Code to verify updates from the Google Safe Browsing API - python

In order to verify the data coming from the Google Safe Browsing API, you can calculate a Message Authentication Code (MAC) for each update. The instructions to do this (from Google) are:
The MAC is computed from an MD5 Digest
over the following information:
client_key|separator|table
data|separator|client_key. The
separator is the string:coolgoog: -
that is a colon followed by "coolgoog"
followed by a colon. The resulting
128-bit MD5 digest is websafe base-64
encoded.
There's also example data to check against:
client key: "8eirwN1kTwCzgWA2HxTaRQ=="
response:
[goog-black-hash 1.180 update][mac=dRalfTU+bXwUhlk0NCGJtQ==]
+8070465bdf3b9c6ad6a89c32e8162ef1
+86fa593a025714f89d6bc8c9c5a191ac
+bbbd7247731cbb7ec1b3a5814ed4bc9d
*Note that there are tabs at the end of each line.
I'm unable to get a match. Please either point out where I'm going wrong, or just write the couple of lines of Python code necessary to do this!
FWIW, I expected to be able to do something like this:
>>> s = "+8070465bdf3b9c6ad6a89c32e8162ef1\t\n+86fa593a025714f89d6bc8c9c5a191ac\t\n+bbbd7247731cbb7ec1b3a5814ed4bc9d\t"
>>> c = "8eirwN1kTwCzgWA2HxTaRQ=="
>>> hashlib.md5("%s%s%s%s%s" % (c, ":coolgoog:", s, ":coolgoog:", c)).digest().encode("base64")
'qfb50mxpHrS82yTofPkcEg==\n'
But as you can see, 'qfb50mxpHrS82yTofPkcEg==\n' != 'dRalfTU+bXwUhlk0NCGJtQ=='.

Anders' answer gives the necessary information, but isn't that clear: the client key needs to be decoded before it is combined. (The example above is also missing a newline at the end of the final table data).
So the working code is:
>>> s = "+8070465bdf3b9c6ad6a89c32e8162ef1\t\n+86fa593a025714f89d6bc8c9c5a191ac\t\n+bbbd7247731cbb7ec1b3a5814ed4bc9d\t\n"
>>> c = "8eirwN1kTwCzgWA2HxTaRQ==".decode('base64')
>>> hashlib.md5("%s%s%s%s%s" % (c, ":coolgoog:", s, ":coolgoog:", c)).digest().encode("base64")
'dRalfTU+bXwUhlk0NCGJtQ==\n'

c="8eirwN1kTwCzgWA2HxTaRQ==".decode('base64')

Related

Python JSONDecoderError

I am not to sure what I am doing wrong. I am trying to parse the specific contents within JavaScript.
This is the output of "s" (for the code below it):
<script type="text/javascript">window._sharedData = {"activity_counts":{"comment_likes":0,"comments":0,"likes":0,"relationships":0,"usertags":0},"config":{"csrf_token":"OIXAF5a6FwMQJj3vCaUQXCGUGL3sFb0Z","viewer":{"allow_contacts_sync":false,"biography":"Follow for the best social media experience. Est. 2014","external_url":null,"full_name":"Social Media Bliztexnetwork","has_profile_pic":true,"id":"6440587166","profile_pic_url":"https://instagram.fbed1-1.fna.fbcdn.net/vp/dd5d8db8ca1645ac8b69fdaf8886184f/5BB11538/t51.2885-19/s150x150/32947488_229940584435561_2806247690365566976_n.jpg","profile_pic_url_hd":"https://instagram.fbed1-1.fna.fbcdn.net/vp/df4d5098687fe594c5b2d9750804941a/5BEC5FC8/t51.2885-19/s320x320/32947488_229940584435561_2806247690365566976_n.jpg","username":"bliztezxxmedia"}},"supports_es6":false,"country_code":"US","language_code":"en","locale":"en_US","entry_data":{"ProfilePage":[{"logging_page_id":"profilePage_7507466602","show_suggested_profiles":false,"graphql":{"user":{"biography":"What a wonderful day!!!","blocked_by_viewer":false,"country_block":false,"external_url":null,"external_url_linkshimmed":null,"edge_followed_by":{"count":17},"followed_by_viewer":true,"edge_follow":{"count":8},"follows_viewer":false,"full_name":"Verna Manning","has_channel":false,"has_blocked_viewer":false,"highlight_reel_count":0,"has_requested_viewer":false,"id":"7507466602","is_private":true,"is_verified":false,"mutual_followers":{"additional_count":-3,"usernames":[]},"profile_pic_url":"https://instagram.fbed1-1.fna.fbcdn.net/vp/96e65311d0a5e79729411bd582592816/5BCC9C5A/t51.2885-19/s150x150/33143922_237271910362316_6290555001760645120_n.jpg","profile_pic_url_hd":"https://instagram.fbed1-1.fna.fbcdn.net/vp/96e65311d0a5e79729411bd582592816/5BCC9C5A/t51.2885-19/s150x150/33143922_237271910362316_6290555001760645120_n.jpg","requested_by_viewer":false,"username":"vernamanning46464","connected_fb_page":null,"edge_felix_combined_post_uploads":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_felix_combined_draft_uploads":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_felix_video_timeline":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_felix_drafts":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_felix_pending_post_uploads":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_felix_pending_draft_uploads":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_owner_to_timeline_media":{"count":2,"page_info":{"has_next_page":false,"end_cursor":"AQAQt_06KHhticevO8Am12l3GJ1CdrZVdUztIDyZN7oXm_IVmr2Clwi844aWh9oe9TU"},"edges":[{"node":{"__typename":"GraphImage","id":"1810494542282448836","edge_media_to_caption":{"edges":[{"node":{"text":"What a sunny day!"}}]},"shortcode":"BkgKzGch1_EsxkqWK-4ZjG_XoWfrFxgXIOrZqs0","edge_media_to_comment":{"count":24},"comments_disabled":false,"taken_at_timestamp":1530047789,"dimensions":{"height":1080,"width":1080},"display_url":"https://instagram.fbed1-1.fna.fbcdn.net/vp/d82d797684ce57fef7a9fe87c74d2342/5BCE0CF2/t51.2885-15/s1080x1080/e15/fr/35274418_207295373248007_2552664476088270848_n.jpg","edge_liked_by":{"count":0},"edge_media_preview_like":{"count":0},"gating_info":null,"media_preview":"ACoqnuL0Qj5cMT0Gf1rBdi5LHqasXLK8hZBtB7fz/WoMV1xjZGLZHijFSbaNtXYVwhgaZtq/n2H1q9/Zn/TRfyNVAxUYHGetNyfU1LT6DuXnsnTn09f6VX21tERy/wAZCjt3P41BNaqMGLkHj1pRl0luJrqjM20m2rrwGM7TjPtzTobYynGQB3J7VpdWv0I8ihtpNtdGkdvGnlvtb1PqaZstPQfmf8az9ouzL5X3RmLMGPIxgdulSiYDgZAPNVQOB/nvSjmvP9pLa5fKibzlPTmjzlH0qMKB27H+ZqOn7SXcXKiwZFAzTPNHpVVSc1ISaPaz7i5Uf//Z","owner":{"id":"7507466602"},"thumbnail_src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/7cecb59edaba9f9f7565604eac28d8df/5BC63210/t51.2885-15/s640x640/sh0.08/e35/35274418_207295373248007_2552664476088270848_n.jpg","thumbnail_resources":[{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/b499ce5fafa113fe57f7325d86628900/5BE96296/t51.2885-15/s150x150/e15/35274418_207295373248007_2552664476088270848_n.jpg","config_width":150,"config_height":150},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/f124ca8254e24569515be5f3f99ff911/5BE9A3A9/t51.2885-15/s240x240/e15/35274418_207295373248007_2552664476088270848_n.jpg","config_width":240,"config_height":240},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/5c82e7c2ae3905863fe25150fca1f5e4/5BCB4ED1/t51.2885-15/s320x320/e15/35274418_207295373248007_2552664476088270848_n.jpg","config_width":320,"config_height":320},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/e76685c6614c444d8ed5f04efc01435a/5BB6D257/t51.2885-15/s480x480/e15/35274418_207295373248007_2552664476088270848_n.jpg","config_width":480,"config_height":480},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/7cecb59edaba9f9f7565604eac28d8df/5BC63210/t51.2885-15/s640x640/sh0.08/e35/35274418_207295373248007_2552664476088270848_n.jpg","config_width":640,"config_height":640}],"is_video":false}},{"node":{"__typename":"GraphImage","id":"1757529388200541080","edge_media_to_caption":{"edges":[{"node":{"text":"What a nice day."}}]},"shortcode":"Bhj_6qyALuYgmy2sPgmUtoBcmcxZWGeyLkM3O00","edge_media_to_comment":{"count":3},"comments_disabled":false,"taken_at_timestamp":1523733851,"dimensions":{"height":1080,"width":1080},"display_url":"https://instagram.fbed1-1.fna.fbcdn.net/vp/16610d58bb6cc90893ffd264f81755c6/5BAD1DBC/t51.2885-15/s1080x1080/e15/fr/30590929_101347367387069_7153309976138612736_n.jpg","edge_liked_by":{"count":1},"edge_media_preview_like":{"count":1},"gating_info":null,"media_preview":"ACoqwgadupijNLn0oAdvNPErDoahopiLkdw6nIPNTfaX9apxDNS7fencLDIAP1qFsZ4qYAx9CDmmMmD1B/GsxkRoqXbnjjP1FN2H2/MVVwHKcVJvqLYf8ml2tSAtbce/vjFIFYeh/Sn0Uhjh7gU8Njjt+dMFLQA4Y9B+HFO+X0/z+VMpaAP/2Q==","owner":{"id":"7507466602"},"thumbnail_src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/a4dfd1c28505301d4c440c95023fbbc7/5BC81D5E/t51.2885-15/s640x640/sh0.08/e35/30590929_101347367387069_7153309976138612736_n.jpg","thumbnail_resources":[{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/d69525a20a2e61b2ee8663daf287a8ee/5BB46AD8/t51.2885-15/s150x150/e15/30590929_101347367387069_7153309976138612736_n.jpg","config_width":150,"config_height":150},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/4034b1752e3a4bace405aadd5a35477c/5BCB79E7/t51.2885-15/s240x240/e15/30590929_101347367387069_7153309976138612736_n.jpg","config_width":240,"config_height":240},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/96b684f38cb826f3efd8a7610ed6e9bb/5BEB899F/t51.2885-15/s320x320/e15/30590929_101347367387069_7153309976138612736_n.jpg","config_width":320,"config_height":320},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/51dac35520bc90d7c7253cc331acf561/5BB44919/t51.2885-15/s480x480/e15/30590929_101347367387069_7153309976138612736_n.jpg","config_width":480,"config_height":480},{"src":"https://instagram.fbed1-1.fna.fbcdn.net/vp/a4dfd1c28505301d4c440c95023fbbc7/5BC81D5E/t51.2885-15/s640x640/sh0.08/e35/30590929_101347367387069_7153309976138612736_n.jpg","config_width":640,"config_height":640}],"is_video":false}}]},"edge_saved_media":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_media_collections":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]}}},"felix_onboarding_video_resources":{"mp4":"/static/videos/felix-onboarding/onboardingVideo.mp4/9d16838ca7f9.mp4","poster":"/static/images/felix-onboarding/onboardingVideoPoster.png/8fdba7cf2120.png"}}]},"gatekeepers":{"ld":true,"rt":true,"sw":true,"vl":true,"seo":true,"seoht":true,"2fac":true,"sf":true,"saa":true,"ai":true},"knobs":{"acct:ntb":0,"cb":0,"captcha":0},"qe":{"dash_for_vod":{"g":"","p":{}},"aysf":{"g":"","p":{}},"bc3l":{"g":"","p":{}},"comment_reporting":{"g":"","p":{}},"direct_conversation_reporting":{"g":"","p":{}},"direct_reporting":{"g":"","p":{}},"reporting":{"g":"","p":{}},"media_reporting":{"g":"","p":{}},"acc_recovery_link":{"g":"","p":{}},"notif":{"g":"","p":{}},"drct_nav":{"g":"","p":{}},"fb_unlink":{"g":"","p":{}},"mobile_stories_doodling":{"g":"","p":{}},"move_comment_input_to_top":{"g":"","p":{}},"mobile_cancel":{"g":"","p":{}},"mobile_search_redesign":{"g":"","p":{}},"show_copy_link":{"g":"control","p":{"show_copy_link_option":"false"}},"mobile_logout":{"g":"","p":{}},"pl_pivot_li":{"g":"control_0423","p":{"show_pivot":"false"}},"pl_pivot_lo":{"g":"","p":{}},"404_as_react":{"g":"","p":{}},"acc_recovery":{"g":"test_with_prefill","p":{"has_prefill":"true"}},"collections":{"g":"","p":{}},"comment_ta":{"g":"","p":{}},"connections":{"g":"control","p":{"has_suggestion_context_in_feed":"false"}},"disc_ppl":{"g":"control_02_27","p":{"has_follow_all_button":"false","has_pagination":"false"}},"embeds":{"g":"","p":{}},"ebdsim_li":{"g":"control_shadow_0322","p":{"is_shadow_enabled":"false","use_new_ui":"true"}},"ebdsim_lo":{"g":"","p":{}},"empty_feed":{"g":"","p":{}},"bundles":{"g":"","p":{}},"exit_story_creation":{"g":"","p":{}},"gdpr_logged_out":{"g":"","p":{}},"appsell":{"g":"","p":{}},"imgopt":{"g":"control","p":{}},"follow_button":{"g":"test","p":{"is_inline":"true"}},"loggedout":{"g":"","p":{}},"loggedout_upsell":{"g":"test_with_new_loggedout_upsell_content_03_15_18","p":{"has_new_loggedout_upsell_content":"true"}},"us_li":{"g":"Test","p":{"show_related_media":"true"}},"msisdn":{"g":"","p":{}},"bg_sync":{"g":"","p":{}},"onetaplogin":{"g":"default_opt_in","p":{"default_value":"true","during_reg":"true","storage_version":"one_tap_storage_version"}},"onetaplogin_userbased":{"g":"","p":{}},"login_poe":{"g":"","p":{}},"prvcy_tggl":{"g":"","p":{}},"private_lo":{"g":"","p":{}},"profile_photo_nux_fbc_v2":{"g":"launch","p":{"prefill_photo":"true","skip_nux":"false"}},"profile_tabs":{"g":"","p":{}},"push_notifications":{"g":"","p":{}},"reg":{"g":"control_01_10","p":{"has_new_landing_appsells":"false","has_new_landing_page":"false"}},"reg_vp":{"g":"","p":{}},"feed_vp":{"g":"launch","p":{"is_hidden":"true"}},"report_haf":{"g":"","p":{}},"report_media":{"g":"","p":{}},"report_profile":{"g":"test","p":{"is_enabled":"true"}},"save":{"g":"test","p":{"is_enabled":"true"}},"sidecar":{"g":"","p":{}},"sidecar_swipe":{"g":"","p":{}},"su_universe":{"g":"test_login_autocomplete","p":{"use_autocomplete_signup":"true"}},"stale":{"g":"","p":{}},"stories_lo":{"g":"test_03_15","p":{"stories_profile":"true"}},"stories":{"g":"","p":{}},"tp_pblshr":{"g":"","p":{}},"video":{"g":"","p":{}},"gdpr_settings":{"g":"","p":{}},"gdpr_blocking_logout":{"g":"","p":{}},"gdpr_eu_tos":{"g":"","p":{}},"gdpr_row_tos":{"g":"test_05_01","p":{"tos_version":"row"}},"fd_gr":{"g":"control","p":{"show_post_back_button":"false"}},"felix":{"g":"test","p":{"is_enabled":"true"}},"felix_clear_fb_cookie":{"g":"control","p":{"is_enabled":"true","blacklist":"fbsr_124024574287414"}},"felix_creation_duration_limits":{"g":"dogfooding","p":{"minimum_length_seconds":"15","maximum_length_seconds":"600"}},"felix_creation_enabled":{"g":"","p":{"is_enabled":"true"}},"felix_creation_fb_crossposting":{"g":"control","p":{"is_enabled":"false"}},"felix_creation_fb_crossposting_v2":{"g":"control","p":{"is_enabled":"true"}},"felix_creation_validation":{"g":"control","p":{"edit_video_controls":"true"}},"felix_creation_video_upload":{"g":"","p":{}},"felix_early_onboarding":{"g":"","p":{}},"pride":{"g":"test","p":{"enabled":"true","hashtag_whitelist":"lgbt,lesbian,gay,bisexual,transgender,trans,queer,lgbtq,girlslikeus,girlswholikegirls,instagay,pride,gaypride,loveislove,pansexual,lovewins,transequalitynow,lesbiansofinstagram,asexual,nonbinary,lgbtpride,lgbta,lgbti,queerfashion,queers,queerpride,queerlife,marriageequality,pride2018,genderqueer,bi,genderfluid,lgbtqqia,comingout,intersex,transman,transwoman,twospirit,transvisibility,queerart,dragqueen,dragking,dragartist,twomoms,twodads,lesbianmoms,gaydads,gendernonconforming"}},"unfollow_confirm":{"g":"","p":{}},"profile_enhance_li":{"g":"control","p":{"has_tagged":"false"}},"profile_enhance_lo":{"g":"control","p":{"has_tagged":"false"}},"create_tag":{"g":"","p":{}}},"hostname":"www.instagram.com","platform":"ios","rhx_gis":"87a25368813608d393baaa28a0d6afb7","nonce":"zsP4NjzdJRIWmer6K5At1A==","zero_data":{},"rollout_hash":"5f72737283f8","bundle_variant":"base","probably_has_app":false,"show_app_install":true};</script>
And this is the code I am trying to execute.
s = str(soup.find_all("script", type="text/javascript")[3])
m = re.search(r"(?<=window._sharedData = )(?P<json>.*)(?=</script>)", s)
if m:
data = json.loads(m.group('json'))
print(data)
for i in data['entry_data']["ProfilePage"]:
for j in i['graphql']['user']['edge_owner_to_timeline_media']['edges']:
print(j['node']["id"])
Upon running this, I am prompted with the following error:
json.decoder.JSONDecodeError: Extra data: line 1 column 12215 (char 12214)
I am completely lost and have no idea where I am going wrong. All help is appreciated and thanks to all of those who contribute in advance!
I think you could update your regex to match the json without the semicolon at the end by adding that to the positive lookahead (?=;</script>):
(?<=window\._sharedData = )(?P<json>.*)(?=;</script>)
Your code might look like this without the [3] in the first line for your given example:
s = str(soup.find_all("script", type="text/javascript"))
m = re.search(r"(?<=window\._sharedData = )(?P<json>.*)(?=;</script>)", s)
if m:
data = json.loads(m.group('json'))
for i in data['entry_data']["ProfilePage"]:
for j in i['graphql']['user']['edge_owner_to_timeline_media']['edges']:
print(j['node']["id"])
There's not quite enough here to debug, what you give for s doesn't include the </script> so the pattern never matches when I run it locally, however when I append it, it seems to work correctly
From the error it is clear that the contents of m.group('json') is not actually a valid JSON string so I suspect you need to work on your regular expression. Try printing out the value of m.group('json') (before attempting to parse it) and feeding that into a a json validator such as https://jsonlint.com/ which will direct you to where the error lies, perhaps that line terminates with a ; that you need to strip out or some other issue

Zip Password functionality works wrong

I have a paradoxon, that I just cannot explain.
In short: I built a python script that is supposed to crack a zipped file, which is password protected.
This is what I've done:
(1) zip the text file:
zip --password bla zip3.zip myZip So the passphrase is "bla".
(2) Then I use the following Python Script:
import zipfile
import itertools
from itertools import *
import string
import time
That's the basic function, that is supposed to check, if a given password works or not:
def crack(File, pwd):
try:
File.extractall(pwd=str.encode(pwd))
print("\n---- SUCCESS! {0} ----".format(pwd))
except:
print("{0} did not work.".format(pwd))
pass
Here I specify, which characters I want to use for trying:
myLetters = string.ascii_letters
Here I specify, which zip-file I want to crack:
File = zipfile.ZipFile("PATH/TO/MY/zip3.zip", 'r')
Here I specify, how long the password-phrase is:
pwd_len = 3
here I specify, how many possible combinations of the charactes exist:
all_poss = (len(myLetters)**pwd_len)
Here is the procedure for concrete password cracking:
count = 0
start_time = time.time()
for i in range(0,pwd_len+1):
for j in map(''.join, itertools.product(myLetters, repeat=i)):
crack(File, j)
count += 1
print(round((count/all_poss)*100, 1), end='\r')
res_time = time.time() - start_time
print("\n--- {} ---".format(round(res_time,2)))
I use a nested loop, to try every password. If it works, I should get the Success-message. Else I should only see the "doesn't work message".
However...
If I type in my terminal: python3 pwdCracker.py >> out I get a long text file, which contains many many "does not work messages", BUT I also get a whole bunch of "Success-messages", although only ONE ("bla") should be correct.
Here is a little extract:
wN did not work.
---- SUCCESS! wO ----
wO did not work.
wP did not work.`
So apparently "wO" is working.. But why?? I set the password to "bla"! I really can open the file with "wO"... why can that happen??
Hope you can help!
The default zip encryption is known to be weak, and I think you are seeing hash collisions 1,2.
Most encryption methods (including those used in zip files) need a fixed length key, and so the password is hashed to give that key. The hash function used in zip is crc32 (specified here, although it details a different attack) which was designed for error checking rather than cryptographic hashing. Therefore it will be vulnerable to this type of attack.
The old zip format contains a check byte to quickly verify if your password is right or wrong. This check byte is verified against the last byte of the decrypted 'decryption header'.
Since the check byte is, well, only one byte, false positives happen quite frequently (1/256). What bruteforce crackers usually do in these cases is to check against multiple files from the same archive (hence using multiple check bytes).
From PkWare's APPNOTE.TXT:
After the header is decrypted, the last 1 or 2 bytes in Buffer SHOULD
be the high-order word/byte of the CRC for the file being decrypted,
stored in Intel low-byte/high-byte order. Versions of PKZIP prior to
2.0 used a 2 byte CRC check; a 1 byte CRC check is used on versions after 2.0.
This can be used to test if the password supplied is correct or not.
So what you are seeing is just that, false positives.

What causes the discrepancy from HMAC functions in R vs Python when using .digest()?

I am trying to write some functions to connect to IBM Cloud Object Storage using AWS Version 4 signing protocols. There's some boilerplate code for doing this that works in Python over here, but when I try to convert the code to R I find myself in a conundrum.
When using HMAC functions in R & Python the .hexdigest() method yields identical outputs for the same string in both languages:
# Python
import hmac
import hashlib
>>> hmac.new("key", "message".encode('utf-8'), hashlib.sha256).hexdigest()
'6e9ef29b75fffc5b7abae527d58fdadb2fe42e7219011976917343065f58ed4a
# R
library(digest)
> hmac("key", enc2utf8("message"), "sha256")
[1] "6e9ef29b75fffc5b7abae527d58fdadb2fe42e7219011976917343065f58ed4a"
If you use the .digest() method in Python from the crypto library - which the code from AWS/IBM suggests - then you get ASCII output. I am able to approximate this ASCII output in R by changing the hmac() output to raw and then converting to character. Below you can see it is not exactly the same - though it is close.
# Python
>>> hmac.new("key", "message".encode('utf-8'), hashlib.sha256).digest()
"n\x9e\xf2\x9bu\xff\xfc[z\xba\xe5'\xd5\x8f\xda\xdb/\xe4.r\x19\x01\x19v\x91sC\x06_X\xedJ"
# R
> rawToChar(hmac("key", "message", "sha256", raw = T))
[1] "n\x9e\xf2\x9bu\xff\xfc[z\xba\xe5'Տ\xda\xdb/\xe4.r\031\001\031v\x91sC\006_X\xedJ"
What is causing the discrepancy here? The source code and docs for .digest() says that it returns a string of 8-bit data that may contain non-ASCII bytes. How can I replicate this output in R?
Ultimately I believe this is what is causing my signatures to fail when trying to authenticate with IBM Cloud Object Storage. Any help or alternative suggestions for HMAC libraries in Python/R would be greatly appreciated!

Why does 'itsdangerous' module return the signed text?

I'm wondering if anyone tell me why the 'itsdangerous' module returns the signed text as a portion of the string. As in:
>>> import itsdangerous
>>>
>>> secret = 'my-secret-key'
>>> token = itsdangerous.TimedSerializer(secret)
>>> token.dumps('my-user-id')
'"my-user-id".Cj51kA.yuoSx6eK0LuuphWK0TlOBil2PM0'
I supposed I could just do something like this to get the hash:
token.dumps('my-user-id').split('.', 1)[1]
... but I'm surprised that I would even need to do this in the first place. The fact that the documentation doesn't explicitly mention this behavior or simply offer a method to strip out the signed text makes me nervous enough to question whether I'm doing something insecure. Thanks in advance for shedding light on the following questions:
1) Is there a good reason why the library would do this?
2) What is the safest way to ensure I don't return the encoded string in plain text along with the hash?
The purpose of itsdangerous is not encrypting your data, it just a simple tool to detect tampered data.
... When you get the data back you can easily ensure that nobody
tampered with it. 1
Therefore, you should encrypt it yourself, before or after signing it by this module.
itsdangerous signs the text or any other data so it can be transmitted via unsafe channels and then checked on the other end or upon retrieval from a database where it was store that it wasn't changed/tampered.
So it creates a signature, adds it to the signed data and then checks upon retrieval that it wasn't tampered. The other side needs the data and the signature.

Push a raw value to Firebase via REST API

I am trying to use the requests library in Python to push data (a raw value) to a firebase location.
Say, I have urladd (the url of the location with authentication token). At the location, I want to push a string, say International. Based on the answer here, I tried
data = {'.value': 'International'}
p = requests.post(urladd, data = sjson.dumps(data))
I get <Response [400]>. p.text gives me:
u'{\n "error" : "Invalid data; couldn\'t parse JSON object, array, or value. Perhaps you\'re using invalid characters in your key names."\n}\n'
It appears that they key .value is invalid. But that is what the answer linked above suggests. Any idea why this may not be working, or how I can do this through Python? There are no problems with connection or authentication because the following works. However, that pushes an object instead of a raw value.
data = {'name': 'International'}
p = requests.post(urladd, data = sjson.dumps(data))
Thanks for your help.
The answer you've linked is a special case for when you want to assign a priority to a value. In general, '.value' is an invalid name and will throw an error.
If you want to write just "International", you should write the stringified-JSON version of that data. I don't have a python example in front of me, but the curl command would be:
curl -X POST -d "\"International\"" https://...
Andrew's answer above works. In case someone else wants to know how to do this using the requests library in Python, I thought this would be helpful.
import simplejson as sjson
data = sjson.dumps("International")
p = requests.post(urladd, data = data)
For some reason I had thought that the data had to be in a dictionary format before it is converted to stringified JSON version. That is not the case, and a simple string can be used as an input to sjson.dumps().

Categories

Resources