Develop Reference

Python can't handle Windows long paths with parent directory parts

I'm trying to use .. in a Windows long filename within Python and it is producing unexpected results.
Here is the output from the Python REPL:
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.exists(u'\\\\?\\C:\\BitBucket\\crab6\\product\\installer\\windows\\..\\usage.txt')
False
>>> os.path.exists(u'\\\\?\\C:\\BitBucket\\crab6\\product\\installer\\usage.txt')
True
(I realize the path above isn't over 260 characters and therefore the long path syntax is unnecessary, but for the purposes of this post I've made it more manageable. The outcome is the same with very long paths.)
Has anyone else encountered this, and is it a purposeful limitation of long filenames?

Strange behavior using tempfile.mkdtemp() in virtualenvs when launched in Cygwin

I am seeing inconsistent behavior when creating temporary directories in Python:
# System Python, Windows Console
C:\Python33>python
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.mkdtemp()
'c:\\windows\\temp\\tmpte7fcc'
# Virtualenv Python, Windows Console
Scripts>python
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.mkdtemp()
'c:\\windows\\temp\\tmprziefb'
# System Python, Cygwin Console
$ /cygdrive/c/Python33/python.exe
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.mkdtemp()
'c:\\windows\\temp\\tmprk4fcu'
# Virtualenv Python, Cygwin Console
$ Scripts/python
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.mkdtemp()
'c:\\cygwin\\tmp\\tmppeozcl
The first three cases create temporary directories in C:\Windows\Temp (as expected). Why does the fourth case create temporary directories somewhere else?
EDIT: Additional data requested in the comments:
# System Python, Windows Console
C:\Python33>python
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import tempfile
>>> tempfile._candidate_tempdir_list()
['C:\\WINDOWS\\TEMP', 'C:\\WINDOWS\\TEMP', 'c:\\temp', 'c:\\tmp', '\\temp', '\\tmp', 'C:\\Python33']
>>> [os.environ.get(envname) for envname in ('TMPDIR', 'TEMP', 'TMP')]
[None, 'C:\\WINDOWS\\TEMP', 'C:\\WINDOWS\\TEMP']
# Virtualenv Python, Windows Console
Scripts>python
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import tempfile
>>> tempfile._candidate_tempdir_list()
['C:\\WINDOWS\\TEMP', 'C:\\WINDOWS\\TEMP', 'c:\\temp', 'c:\\tmp', '\\temp', '\\tmp', 'C:\\_PROJECTS\\python-veracity']
>>> [os.environ.get(envname) for envname in ('TMPDIR', 'TEMP', 'TMP')]
[None, 'C:\\WINDOWS\\TEMP', 'C:\\WINDOWS\\TEMP']
# System Python, Cygwin Console
$ /cygdrive/c/Python33/python.exe
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import tempfile
>>> tempfile._candidate_tempdir_list()
['C:\\Cygwin\\tmp', 'C:\\WINDOWS\\TEMP', 'c:\\temp', 'c:\\tmp', '\\temp', '\\tmp', 'C:\\_PROJECTS\\python-veracity']
>>> [os.environ.get(envname) for envname in ('TMPDIR', 'TEMP', 'TMP')]
[None, 'C:\\Cygwin\\tmp', 'C:\\WINDOWS\\TEMP']
# Virtualenv Python, Cygwin Console
$ Scripts/python
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import tempfile
>>> tempfile._candidate_tempdir_list()
['C:\\Cygwin\\tmp', 'C:\\Cygwin\\tmp', 'c:\\temp', 'c:\\tmp', '\\temp', '\\tmp', 'C:\\_PROJECTS\\python-veracity']
>>> [os.environ.get(envname) for envname in ('TMPDIR', 'TEMP', 'TMP')]
[None, 'C:\\Cygwin\\tmp', 'C:\\Cygwin\\tmp']

The tempfile module looks for the environment variables TMPDIR, TEMP and TMP. If one of these variables is set then its value is used as base directory for temporary files and directories. I guess Cygwin sets one of these to C:\cygwin\tmp.

Is there easy way to prevent echo from input?

How to prevent echo from input ??
Have tried "getpass()" but no luck.
On Windows IDLE, it doesn't work
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import getpass
>>> p = getpass.getpass(prompt="Input: ")
Warning: Password input may be echoed.
Input: abc <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< It still echos..
On the terminal of the Windows, it works
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (In
tel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import getpass
>>> p = getpass.getpass(prompt="Input: ")
Input:
>>>
Is there a easy way to prevent echo from input ?

I'm assuming your first example there is in IDLE.
From getpass.win_getpass():
if sys.stdin is not sys.__stdin__:
return fallback_getpass(prompt, stream)
IDLE replaces sys.stdin with a different object. getpass detects that somebody has wrapped stdin and fails for security reasons.
See: http://bugs.python.org/issue9290

Is the Python 3.x signal library for Windows incomplete?

I went to write a system script using 3.0 and found the SIGALRM signal and signal.alarm() call missing amongst many others on the Windows deployment. Does anyone know why these are missing? Below is a dir() of the 2.5 vs 3.0 signal packages on windows. I haven't found any 3.0 docs yet mentioning that this was moved
EDIT: signals do work in python25 on windows, they were removed in 3.0. I should reword my request as 'where did they go or get turned into' or is the windows 3.0 release just not complete yet?
python25> python
Python 2.5.1 (r251:54863, May 18 2007, 16:56:43)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import signal
>>> dir(signal)
['NSIG', 'SIGABRT', 'SIGALRM', 'SIGBUS', 'SIGCHLD', 'SIGCLD',
'SIGCONT', 'SIGEMT', 'SIGFPE', 'SIGHUP', 'SIGILL', 'SIGINT ', 'SIGIO',
'SIGKILL', 'SIGPIPE', 'SIGPOLL', 'SIGPROF', 'SIGQUIT', 'SIGRTMAX',
'SIGRTMIN', 'SIGSEGV', 'SIGSTOP', 'SIGSYS ', 'SIGTERM', 'SIGTRAP',
'SIGTSTP', 'SIGTTIN', 'SIGTTOU', 'SIGURG', 'SIGUSR1', 'SIGUSR2',
'SIGVTALRM', 'SIGWINCH', 'SIGX CPU', 'SIGXFSZ', 'SIG_DFL', 'SIG_IGN',
'__doc__', '__name__', 'alarm', 'default_int_handler', 'getsignal',
'pause', 'sig nal']
>>> exit()
python25> cd ../python31
python31> python
Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import signal
>>> dir(signal)
['NSIG', 'SIGABRT', 'SIGBREAK', 'SIGFPE', 'SIGILL', 'SIGINT',
'SIGSEGV', 'SIGTERM', 'SIG_DFL', 'SIG_IGN', '__doc__', '__ name__',
'__package__', 'default_int_handler', 'getsignal', 'set_wakeup_fd',
'signal']
>>>

It seems you are running your 2.5 in cygwin, which is probably the reason that it shows up there.
Python 2.5.1 (r251:54863, May 18 2007, 16:56:43)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
In my 2.5 for win32 it looks just like in your 3.1:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import signal
>>> dir(signal)
['NSIG', 'SIGABRT', 'SIGBREAK', 'SIGFPE', 'SIGILL', 'SIGINT', 'SIGSEGV', 'SIGTERM', 'SIG_DFL', 'SIG_IGN', '__doc__', '__
name__', 'default_int_handler', 'getsignal', 'signal']

Windows is NOT posix compliant OS so it does not have all signals - my guess is that on 3.0 the missing signals do not show up there any longer.

Why do I get wrong results for hmac in Python but not Perl?

I'm trying to compute hmac using sha-512.
The Perl code:
use Digest::SHA qw(hmac_sha512_hex);
$key = "\x0b"x20;
$data = "Hi There";
$hash = hmac_sha512_hex($data, $key);
print "$hash\n";
and gives the correct hash of
87aa7cdea5ef619d4ff0b4241a1d6cb02379f4e2ce4ec2787ad0b30545e17cde
daa833b7d6b8a702038b274eaea3f4e4be9d914eeb61f1702e696c203a126854
Python version:
import hashlib, hmac
print hmac.new("\x0b"*20, "Hi There", hashlib.sha512).hexdigest()
which gives the incorrect hash of
9656975ee5de55e75f2976ecce9a04501060b9dc22a6eda2eaef638966280182
477fe09f080b2bf564649cad42af8607a2bd8d02979df3a980f15e2326a0a22a
any ideas why the Python version is giving me the wrong hash?
Edit:
version is
Python 2.5.1 (r251:54863, Jan 13 2009, 10:26:13)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin

yes indeed -- it seems the Leopard version of python2.5 is the one that is broken.
below run on a Penryn-based MBP...
$ **uname -a**
Darwin lizard-wifi 9.6.0 Darwin Kernel Version 9.6.0: Mon Nov 24 17:37:00 PST 2008; root:xnu-1228.9.59~1/RELEASE_I386 i386
dpc#lizard-wifi:~$ **which python**
/usr/bin/python
Running this version installed in Leopard OS
dpc#lizard-wifi:~$ python
Python 2.5.1 (r251:54863, Jan 13 2009, 10:26:13)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib, hmac
>>> print hmac.new("\x0b"*20, "Hi There", hashlib.sha512).hexdigest()
9656975ee5de55e75f2976ecce9a04501060b9dc22a6eda2eaef638966280182477fe09f080b2bf564649cad42af8607a2bd8d02979df3a980f15e2326a0a22a
>>>
And then the MacPorts version of python2.5
$ /opt/local/bin/python2.5
Python 2.5.4 (r254:67916, Feb 3 2009, 21:40:31)
[GCC 4.0.1 (Apple Inc. build 5488)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib, hmac
>>> print hmac.new("\x0b"*20, "Hi There", hashlib.sha512).hexdigest()
87aa7cdea5ef619d4ff0b4241a1d6cb02379f4e2ce4ec2787ad0b30545e17cdedaa833b7d6b8a702038b274eaea3f4e4be9d914eeb61f1702e696c203a126854
>>>

I am unable to replicate your results here. In IDLE using Python 2.5:
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information.
...
IDLE 1.2.2
>>> import hashlib, hmac
>>> print hmac.new("\x0b"*20, "Hi There", hashlib.sha512).hexdigest()
87aa7cdea5ef619d4ff0b4241a1d6cb02379f4e2ce4ec2787ad0b30545e17cdedaa833b7d6b8a702038b274eaea3f4e4be9d914eeb61f1702e696c203a126854

Which version of Python? Strings are Unicode in Python 3. Is this a Unicode issue?

Under python 2.5.2 I get the correct hash
I guess the old version was the problem