Develop Reference

Python Download Roblox Data

I'm attempting to format json from the Roblox API. I have a names.txt that stores all of the names. This is how the file looks
rip_robson0007
Abobausrip
app_58230
kakoytochelik123
Ameliathebest727
Sherri0708
HixPlayk
mekayla_091
ddddorffg
ghfgrgt7nfdbfj
TheWolfylikedog
paquita12345jeje
hfsgfhsgfhgfhds
It stores a bunch of usernames seperated by a new line. The code is suppose to use the names and for each name get the JSON from this endpoint https://api.roblox.com/users/get-by-username?username={name} & format it as I have in my code. It always returns error 429 and doesn't save any of the data.
This is the code:
import json
import requests
import time
# Read the names from the text file
with open("./txt/names.txt", "r") as f:
names = f.read().split("\n")
# Initialize an empty dictionary to store the users
users = {}
# Iterate through the names
for name in names:
time.sleep(5)
response = requests.get(f"https://api.roblox.com/users/get-by-username?username={name}")
# Check the status code of the response
if response.status_code != 200:
print(f"Failed to get data for {name}: {response.status_code}")
continue
# Try to parse the response as JSON
try:
user_data = response.json()
except ValueError:
print(f"Failed to parse JSON for {name}")
continue
# Extract the necessary information from the response
user_id = user_data["Id"]
username = user_data["Username"]
avatar_uri = user_data["AvatarUri"]
avatar_final = user_data["AvatarFinal"]
is_online = user_data["IsOnline"]
# Add the user's information to the dictionary
users[user_id] = {
"Id": user_id,
"Username": username,
"AvatarUri": avatar_uri,
"AvatarFinal": avatar_final,
"IsOnline": is_online
}
# Save the dictionary to a JSON file
with open("users.json", "w") as f:
json.dump(users, f)

You can often overcome this with judicious use of proxies.
Start by getting a list of proxies from which you will make random selections. I have a scraper that acquires proxies from https://free-proxy-list.net
My list (extract) looks like this:-
http://103.197.71.7:80 - no
http://163.116.177.33:808 - yes
'yes' means that HTTPS is supported. This list currently contains 95 proxies. It varies depending on the response from my scraper
So we start by parsing the proxy list. Subsequently we choose proxies at random before trying to access the Roblox API. This may not run quickly because the proxies are not necessarily reliable. They are free after all.
from requests import get as GET, packages as PACKAGES
from random import choice as CHOICE
from concurrent.futures import ThreadPoolExecutor as TPE
PACKAGES.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ALL:#SECLEVEL=1'
ROBLOX_API = 'https://api.roblox.com/users/get-by-username'
TIMEOUT = 1
def get_proxies():
http, https = list(), list()
with open('proxylist.txt') as p:
for line in p:
proxy_url, _, supports_https = line.split()
_list = https if supports_https == 'yes' else http
_list.append(proxy_url)
return http, https
http, https = get_proxies()
def process(name):
params = {'username': name.strip()}
while True:
try:
proxy = {'http': CHOICE(http), 'https': CHOICE(https)}
(r := GET(ROBLOX_API, params=params, proxies=proxy, timeout=TIMEOUT)).raise_for_status()
if (j := r.json()).get('success', True):
print(j)
break
except Exception as e:
pass
with open('names.txt') as names:
with TPE() as executor:
executor.map(process, names)
In principle, the while loop in process() could get stuck so it might make sense to limit the number of retries.
This produces the following output:
{'Id': 4082578648, 'Username': 'paquita12345jeje', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 2965702542, 'Username': 'mekayla_091', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 4079018794, 'Username': 'app_58230', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 3437922948, 'Username': 'kakoytochelik123', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 4082346906, 'Username': 'Abobausrip', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 2988555289, 'Username': 'HixPlayk', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 3286921649, 'Username': 'Sherri0708', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 1441252794, 'Username': 'ghfgrgt7nfdbfj', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 4088896225, 'Username': 'ddddorffg', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 3443374919, 'Username': 'TheWolfylikedog', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 3980932331, 'Username': 'Ameliathebest727', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 3773237135, 'Username': 'rip_robson0007', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}
{'Id': 4082991447, 'Username': 'hfsgfhsgfhgfhds', 'AvatarUri': None, 'AvatarFinal': False, 'IsOnline': False}

Trying to make a local file dictionary and search system

I am trying to make a local file searcher, which will search for files based on tags, and also will by names. i dont have any idea on how to make the searching system nor the python dictionary and searching with tags which confuse me.
files = {'samplefile1.txt', 'samplefile2.txt'}
fileName = ''
fileDiscription = 'Enter Discription here'
isMP3File = True
isMP4File = True
isTxtFile = True
isArchived = True
tags = ['sample1', 'sample2', 'favorited']
filesDictionary = {
'samplefile1.txt': {
fileName: 'coolFile1',
fileDiscription: 'cool disc.',
isMP3File: False,
isMP4File: False,
isTxtFile: True,
isArchived: False,
tags = ['sample1', 'favorited']
},
'samplefile1.txt': {
fileName: 'coolFile2',
fileDiscription: 'cool disc2',
isMP3File: False,
isMP4File: False,
isTxtFile: True,
isArchived: True,
tags = ['sample2']
},
}
so in the code above, with search function, it should show only samplefile1.txt when searched by 'sample1', or 'favorited', or samplefile2.txt if searched with 'sample2'
(also fileName is the name i was talking about in this question, not the file name on pc)
(also any idea on how to automate this 'files' dictionary adding using gui (something like how you would post stuff to twitter or smth, with ticks and message boxes))

Create a dictionary where you have each tag as a key, and the filename as a value.
Since you want to search by tag, having the tags as keys will make the search time constant.
searchDict = {
'sample1': ['samplefile1.txt'],
'favorited': ['samplefile1.txt'],
'sample2': ['samplefile2.txt']
}
then given a tag you can just get the filename inmediately
searchDict['sample1'] # will return ['samplefile1.txt']
You can then use that key to access your main dictionary files
for filename in searchDict['sample1']:
print(files[filename])
will print
{
fileName: 'coolFile1',
fileDiscription: 'cool disc.',
isMP3File: False,
isMP4File: False,
isTxtFile: True,
isArchived: False,
tags = ['sample1', 'favorited']
}
To create the searchDict, you can iterate once over your database of files, getting the tags and associating them to the filenames. It will be a costly operation if your database is big, but once done your search will run in constant time.

get first element from dictionary which is an array - python

I have an array store header information:
{'x-frame-options': {'defined': True, 'warn': 0, 'contents': 'SAMEORIGIN'}, 'strict-transport-security': {'defined': True, 'warn': 0, 'contents': 'max-age=15552000'}, 'access-control-allow-origin': {'defined': False, 'warn': 1, 'contents': ''}, 'content-security-policy': {'defined': True, 'warn': 0, 'contents': "upgrade-insecure-requests; frame-ancestors 'self' https://stackexchange.com"}, 'x-xss-protection': {'defined': False, 'warn': 1, 'contents': ''}, 'x-content-type-options': {'defined': False, 'warn': 1, 'contents': ''}}
I want to get the first element of the dictionary
#header is a return array that store all header information,
headers = headersecurity.verify_header_existance(url, 0)
for header in headers:
if header.find("x-frame-options"):
for headerSett in header:
defined = [elem[0] for elem in headerSett.values()] # here I don't get first element
print(defined)
the expected results is:
x-frame-options : defined = True;
access-control-allow-origin : defined = True;
x-content-type-options : defined = True;
....
thanks

I think it'd be safer to use dictionary keys like so
headers['x-frame-options']['defined']
That way you do not rely on the ordering inside the dict (dict are not ordered)
EDIT: Just saw your edit and what you are expecting as an output, here a simple way of having it:
for key, value in headers.items():
if "defined" in value:
print(f"{key} : defined = {value['defined']}")
output:
x-frame-options : defined = True
strict-transport-security : defined = True
access-control-allow-origin : defined = False
content-security-policy : defined = True
x-xss-protection : defined = False
x-content-type-options : defined = False

Choosing an equlation from api response Python

Im trying to work with api responses;
Here is the example response that comes from api;
{u'blog': {u'followed': False, u'is_adult': False, u'can_subscribe': False, u'is_nsfw': False, u'ask': True, u'likes': 920, u'is_blocked_from_primary': False, u'can_submit': True, u'ask_anon': True, u'subscribed': False, u'share_likes': True, u'updated': 1493576375, u'description': u'<p>"Che hai dei bellissimi occhi quando mi cerchi."</p><p>18 </p><p>Beginner Wiccan and Witch </p><p>\U0001f312\U0001f315\U0001f318</p>', u'total_posts': 13992, u'submission_page_title': u'Submit', u'submission_terms': {u'title': u'Submit', u'tags': [], u'guidelines': u'', u'accepted_types': [u'text', u'photo', u'quote', u'link', u'video']}, u'name': u'darknessinmyheartt', u'url': u'http://darknessinmyheartt.tumblr.com/', u'ask_page_title': u'lets ask something!/ haydi sor!', u'title': u'"Laurel"', u'posts': 13992, u'reply_conditions': u'3', u'can_send_fan_mail': False}}
how can ı get only the value of u'updated' from that response
u'updated': 1493576375
I have to define that value to "x"

I think the tumblr api response is a python dict, based on that, try:
x = client.blog_info('darknessinmyheartt')
print x['blog']['updated']

You got json response, in Python you can get value from json data, via key.
For your example, you need to do next things:
your_data['blog']['updated']
via your_data['blog'] You will get object with key, values
{'followed': False, u'is_adult': False, u'can_subscribe': False, ....}
and via your_data['blog']['updated'] you will get value 1493576375

Find and Replace text within headers with Win32COM

I'd like to find some words in the headers of a Word document and replace them with other words : I've done this in the body of the document with the following code, and it works fine.
import win32com.client
wdFindContinue = 1
wdReplaceAll = 2
app = win32com.client.DispatchEx("Word.Application")
app.Visible = 1
app.DisplayAlerts = 0
app.Documents.Open(document_path)
FromTo = {"<#TITLE#>":"My title", "<#DATE#>":"Today"}
for From in FromTo.keys():
app.Selection.Find.Execute(From, False, False, False, False, False, True, wdFindContinue, False, FromTo[From], wdReplaceAll)
The problem is that this code doesn't work for headers and footers. I've also tried this :
app.ActiveDocument.Sections(1).Headers(win32com.client.constants.wdHeaderFooterPrimary).Range.Select
app.Selection.Find.Execute(From, False, False, False, False, False, True, wdFindContinue, False, FromTo[From], wdReplaceAll)
But it doesn't work better (despite the fact that I don't have any error message).
Does someone have an idea on how to do that? Another information is that I have an image inserted in the headers as well, I don't know if it matters or not.

You must activate header/footer pane after open document.
Language Visual basic. Change syntax to python
ActiveDocument.ActiveWindow.Panes(1).View.SeekView=wdSeekCurrentPageHeader
for header and
ActiveDocument.ActiveWindow.Panes(1).View.SeekView = wdSeekCurrentPageFooter
for footer
Then search/replace
To change pane to main part use
ActiveDocument.ActiveWindow.Panes(1).View.SeekView = wdSeekMainDocument