Develop Reference

Python module 'modules' has no attribute 'millis'

Completely new to Python, I have installed the latest stock version, updated PIP and tried to run a script been given to import data from API (person provided says works for them), when tried to run I have installed each library as required.
import json
import pandas as pd
from pandas.io.json import json_normalize
import requests
import modules
from requests_oauthlib import OAuth1
from datetime import datetime
this is main header, I am now getting error AttributeError: module 'modules' has no attribute 'millis' on following line
payload = {'interactive': 'true',
"ended": 'true',
"start": {"from": modules.millis(2019,4,21,0,0), # CHANGE HERE THE DATE
"to": modules.millis(2019,4,24,23,59)}, # CHANGE HERE THE DATA
'skillIds':['1286977632']
}
Any ideas.

sorry guys I have found the fix, it was in a separate file I was not given at the time
# returns the elapsed milliseconds since the start of the program
def millis(year,month,day,hour,minute):
dt = datetime(year,month,day,hour, minute) - start_time
ms = (dt.days * 24 * 60 * 60 + dt.seconds) * 1000 + dt.microseconds / 1000.0
return round(ms)

Datetime and Time can't be used in the same program?

So I'm writing a program with
import datetime
import time
I'm using time to record the time it takes the program to run, and I need to check the date so if the file is more than a certain age, don't process it.
I keep getting this error when trying to use these two classes
Traceback (most recent call last):
File "<stdin>", line 563, in <module>
File "<stdin>", line 498, in main
AttributeError: type object 'datetime.time' has no attribute 'time'
shell returned 1
Is it not possible to use both time and datetime in one program?
Some of the code:
import PyPDF2
import re
import os
#Time testing
import time
#Using this to check if address is in proper format and to clean it up
import usaddress
#testing this one out
import datetime
from dateutil.parser import *
from dateutil.tz import *
from datetime import *
#Timer starts
start_time = time.time() #Error is referring to this line, line 498
#Opens 3 different files
#For file in folder parse it to text
#Writes some things to file
#Gets the date from the file
if date != None:
fileDate = parse(date).year
now = datetime.now()
print now.year, now.month, now.day
#Ends the timer and prints the time to the console
print("--- %s seconds ---" % round(time.time() - start_time, 2))

Here is your problem:
import datetime
from dateutil.parser import *
from dateutil.tz import *
from datetime import * # <<<< problems
First you are importing datetime and then you are importing everything from datetime.
Be explicit, and only import what you need.
from datetime import datetime
Then you can use it as datetime.now or whatever methods you may need.
As a rule of thumb, never import *. It causes exactly these sorts of issues.

The problems is:
from datetime import *
because it imports time from datetime. It's always better to import only what you need. But if you also really need this method you could do (for example):
from datetime import time as dt
That's why import * is dangerous...

importing module (nltk) causes multiprocessing to hang

I tracked a python multiprocessing headache down to the import of a module (nltk). Reproducible (hopefully) code is pasted below. This doesn't make any sense to me, does anybody have any ideas?
from multiprocessing import Pool
import time, requests
#from nltk.corpus import stopwords # uncomment this and it hangs
def gethtml(key, url):
r = requests.get(url)
return r.text
def getnothing(key, url):
return "nothing"
if __name__ == '__main__':
pool = Pool(processes=4)
result = list()
nruns = 4
url = 'http://davidchao.typepad.com/webconferencingexpert/2013/08/gartners-magic-quadrant-for-cloud-infrastructure-as-a-service.html'
for i in range(0,nruns):
# print gethtml(i,url)
result.append(pool.apply_async(gethtml, [i,url]))
# result.append(pool.apply_async(getnothing, [i,url]))
pool.close()
# monitor jobs until they complete
running = nruns
while running > 0:
time.sleep(1)
running = 0
for run in result:
if not run.ready(): running += 1
print "processes still running:",running
# print results
for i,run in enumerate(result):
print i,run.get()[0:40]
Note that the 'getnothing' function works. It's a combination of the nltk module import and the requests call. Sigh
> python --version
Python 2.7.6
> python -c 'import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)'
('7fffffffffffffff', True)
> pip freeze | grep requests
requests==2.2.1
> pip freeze | grep nltk
nltk==2.0.4

I would redirect others with similar problems to solutions which do not use the multiprocessing module:
1) Apache Spark for scalability/flexibility. However, this doesn't seem to a solution for python multiprocessing. Looks like pyspark is also limited by the Global Interpreter Lock?
2) 'gevent' or 'twisted' for general python asynchronous processing
http://sdiehl.github.io/gevent-tutorial/
3) grequests for asynchronous requests
Asynchronous Requests with Python requests

Why does python not import every module at startup automatically?

I was having a play around with Python 2.7 and everybody knows that at the start of every program, you always have to import modules. For example:
import random
import time
for x in range(1, 300):
print random.randint(1,100)
time.sleep(1)
print "Done!"
Anyway, I was thinking, why do I have to import all my modules manually? Why doesn't Python just import them all like this.
Sure, I can understand why it does not import like this:
from random import randint
from time import *
for x in range(1, 300):
print randint(1,100)
sleep(1)
print "Done!"
As some function names may clash. But, if you have to define where the function is at the start, so for example random. in random.randint(1,100).
Now modern computers are so powerful, it seems logical to import every module automatically instead of wasting lines of code, and time by having to find which module you need then importing it manually when it can easily be automated.
So, why does python not import every module at startup automatically?
EDIT:
I have made a new version of a little program that imports every module that I can find by running:
import sys
sys.builtin_module_names
Here are the results:
x = int(1000000)
def test():
global x
x -= 1
print "Iterations Left: ", x
import __builtin__
import __main__
import _ast
import _bisect
import _codecs
import _codecs_cn
import _codecs_hk
import _codecs_iso2022
import _codecs_jp
import _codecs_kr
import _codecs_tw
import _collections
import _csv
import _functools
import _heapq
import _hotshot
import _io
import _json
import _locale
import _lsprof
import _md5
import _multibytecodec
import _random
import _sha
import _sha256
import _sha512
import _sre
import _struct
import _subprocess
import _symtable
import _warnings
import _weakref
import _winreg
import array
import audioop
import binascii
import cPickle
import cStringIO
import cmath
import datetime
import errno
import exceptions
import future_builtins
import gc
import imageop
import imp
import itertools
import marshal
import math
import mmap
import msvcrt
import nt
import operator
import parser
import signal
import strop
import sys
import thread
import time
import xxsubtype
import zipimport
import zlib
def start():
from timeit import Timer
t = Timer("test()", "from __main__ import test")
print t.timeit()
start()

Because you don't need all of it. There is no point in loading every library if you don't need them.
EDIT:
I copied my libs folder to a test directory and made it into a package by adding an __init__.py file to it. In this file I added:
import os
import glob
__all__ = [ os.path.basename(f)[:-3] for f in glob.glob(os.path.dirname(__file__)+"/*.py")]
I created a test script that contains:
from Lib import *
print('Hello')
When I try to run it in the shell all it does is print 'The Zen of Python' by Tim Peters, opens this webcomic in my browser (2 things I absolutely did not see coming) and throws the following error:
Traceback (most recent call last):
File "C:\Users\Hannah\Documents\dropBox\Python\test\test.py", line 1, in <module>
from Lib import *
AttributeError: 'module' object has no attribute 'crypt'
It takes a noticable amount of time before it does any of this, about 10-15 seconds

Maybe what you would like is a feature that automatically imports the libraries that are used in your script without needing to specify them at the beginning. I found this on the Internet http://www.connellybarnes.com/code/autoimp/
You just need one import at the beginning of your script
from autoimp import *
All other modules are loaded "lazily", i.e. when they are first used.
Example in the interactive shell:
>>> random.random()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'random' is not defined
>>> from autoimp import *
>>> random.random()
0.0679000238267422
From the docs:
For ultimate laziness, place the command "from autoimp import *" in your PYTHONSTARTUP file. Now your interactive session has all modules available by default.

Every module you import takes time to import. Importing every built-in module every time you start Python would kill performance in a lot of important scenarios where new Python interpreters are started frequently.

Python does have a set of modules that are always loaded, its call __builtins__ :).
Python's builtins provide the import statement for you to extend your scope with even more modules! But as other posts have said, deciding your script needs these modules it up to you. -- I have looked into mutating __builtins__ and I promise you, explicitly importing what you need is the better option.
((Big rant about not using from name import * cut from here))
Since most of writing python ultimately becomes packaging and installing that writen python somewhere, this is my goto set of resources for getting a handle on python's infamous import:
Start by sticking to standard tools and libraries (https://packaging.python.org/current/)
Reading and understand The Google Python Standards Guide (https://google.github.io/styleguide/pyguide.html),
Read the Zen of Python (https://www.python.org/dev/peps/pep-0020/)
Be Pythonic (basically adhere to "The Zen of Python"), https://www.youtube.com/watch?v=wf-BqAjZb8M
Supplement your problem-space with tips from The Hitchhikers Guid to Python (http://docs.python-guide.org/en/latest/)
Be preapred to package your code (https://packaging.python.org/distributing/ (Doc), https://github.com/pypa/sampleproject/ (Example))
Being prepared to debug someone else's and, Your own code by getting familiar with tools like:
pdb (import pdb; pdb.set_trace(), > pp variable),
print(help(variable)),
dir(variable),
and pprint.pprint( variable.__dict__ )

Launching a .exe Upon a Timer's Due_Date

How Would i Go About Adding a File To launch when it see that's it's the due_date? I've try'd quite i few different method's from Google but i'm still having a hard time figuring it out. currently it's set to wait 36 hours after launching the .py file.
any help would be great and it'd get this monkey off my back for good!
import datetime
import croniter
import crontab
import time
c = croniter.croniter("0 9,10,11 * * TUE")
next_due_date = c.get_next(datetime.datetime)
while True:
now = datetime.datetime.now()
if now > next_due_date:
do_something(line.py)
time.sleep(60 * 60 * 36)
else:
time.sleep(60 * 60 * 2)

If it's a .exe you can just use os.system("myexecutable.exe") after launching the python
import datetime
import croniter
import crontab
import time
c = croniter.croniter("0 9,10,11 * * TUE")
next_due_date = c.get_next(datetime.datetime)
while True:
now = datetime.datetime.now()
if now > next_due_date:
do_something(line.py) # Edit: fixed tabbing; just in case it wasn't tabbed in
# your script
# Use os.system to run the exe
os.system("myexecutable.exe")
time.sleep(60 * 60 * 36)
else:
time.sleep(60) # Edit: I always find that it's better to have a smaller
# sleep time
You can also use the subprocess module so you can halt the script or track if the exe is still running instead.