Develop Reference

How do I write to a file with spaces and special characters in file name?

I'm trying to write to a file called
data/STO: INVE-B-TIME_SERIES_DAILY_ADJUSTED-1.csv
using the following python code on windows
overwrite = "w" # or write new
f = open(OutputPath(copyNr), overwrite)
f.write(response.content.decode('utf-8'))
f.close()
print(f"Wrote to file: {OutputPath(copyNr)}")
The output in the console is correct, but the file written to results in only data/STO so the path seems to be clipped. I've tried to escape the characters using the method from this SO answer but that gave me an invalid_argument exception for the following filename:
data/STO\\:\\ INVE-B-TIME_SERIES_DAILY_ADJUSTED-1.csv
I get the same error when I remove the space and it still seems to clip at :. How do I include such characters in a filename?

You can't. Colons are only allowed on the volumeseperator - anywhere else there are illegal.
Allowed:
d:/somedir/somefile.txt
Illegal:
./some:dir/somefile.txt
and as seperators you can either use '\\' or '/' - both should be understood.
This is not a python limitation but a operating system limitation.
See f.e. What characters are forbidden in Windows and Linux directory names?

Windows API will not allow colon in filename, it is reserved for drive letter, use a different sign.

Concatenating string and filepath adds extra backslash in pandas

This is almost embarrassingly simple, but I couldn't find a way around this at all. My code used to work just fine as well, and nothing fundamental has changed. Honestly, I can't see the mistake.
In the following code whenever I run a pd.read_excel, I get an extra backslash in my concatenation:
sports_data = [pd.read_excel(r'Data\NData' + str(season) + ".xlsx") for season in season_list]
All I want is the filepath beginning, to be tied to the season variable I set earlier in a list.
Thanks, and trust me that I checked high and wide for this!
I've tested the same with double slashes, as well as a raw string. None of them work as each time the backslashes are doubled. I've even separated the filepath, but it still doesn't work
The error I get is the following, each time:
FileNotFoundError: [Errno 2] No such file or directory: 'Data\\NData2002-2003.xlsx'
If I attempt to run a double-backslash to get the line skip to be a simple backslash, it is still doubled:
FileNotFoundError: [Errno 2] No such file or directory: 'Data\\\\NData2002-2003.xlsx'

This behavior is consistent with Python's normal behavior. Try it out in the REPL:
>>> s = r'path\to\my\file'
>>> s
'path\\to\\my\\file'
>>> print(s)
path\to\my\file
A raw string literal still gets stored as a normal string internally, and backslashes still need to be escaped in Python when getting a filepath. Are you sure that the file exists? Double-check the filename and type to see that they match exactly.

'invalid argument' error opening file (and not reading file)

I am trying to write code that takes 2 numbers in a text file and then divides them, showing the answer as a top heavy fraction. I have gotten the fractions part to work when I am inputting my own values in the program, but i cannot get the program to recognise the text file. I have tried putting them in the same directory and putting the full system path of the file, but nothing so far has worked. right now I am just trying to get the contents of the file to print.
with open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w') as f:
for line in f:
for word in line.split():
print(word)
I will then assign the 2 values to x and y, but I get this error:
Traceback (most recent call last):
File "C:\Python34\divider.py", line 2, in <module>
open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w')
OSError: [Errno 22] Invalid argument:'C:\\ProgramData\\Microsoft\\Windows\\Startmenu\\Programs\\Python 3.4\topheavy.txt'

open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w')
OSError: [Errno 22] Invalid argument:'C:\\ProgramData\\Microsoft\\Windows\\Startmenu\\Programs\\Python 3.4\topheavy.txt'
Two things:
When working with paths that contain backslashes, you either need to use two backslashes, or use the r'' form to prevent interpreting of escape sequences. For example, 'C:\\Program Files\\...' or r'C:\Program Files\...'.
Your error shows this: \\Startmenu\\. It appears that a space is missing between "Start" and "menu", despite the fact that the open line seems to have the right path.
Note: that the \topheavy.txt in your path is probably getting converted to <tab>opheavy.txt too. That's why there aren't two backslashes in front of it in the traceback.

You are using a "\" separator which is probably getting escaped somewhere (like that \t near the end. That's the Windows path separator, but also used as a string escape.
You can double up the "\" as "\". Easiest however is to prepend an r at the beginning to ignore .
r"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt"
Skip the recommendation to use / instead, you are not on Unix and there is no reason Python can't accommodate Windows, as long as you remember to take care about "\" also being an escape. Using r' at start also allows you to copy/paste from the string into another program or vice-versa.
also, it wouldn't hurt to test in c:\temp or similar to avoid issues where you may have mistyped your path.
Last, but not least, you need to open in "r" read mode, as previously mentioned.

You should add one more "/" in the last "/" of path for example:
open('C:\Python34\book.csv') to open('C:\Python34\\\book.csv')
Reference

I had this same error appear when trying to read a large file in Python 3.5.4. To solve it, instead of reading the whole file into memory with .read(), I read each line one by one:
with open('big.txt') as f:
for i in f:
print(i)

Just as is written on the Python Documentation, the IOError Exception occurs:
Raised when an I/O operation (such as a print statement, the built-in
open() function or a method of a file object) fails for an I/O-related
reason, e.g., “file not found” or “disk full”.

Open with "r" (read) instead of "w" (write)
And startmenu in these two lines are different?? Try using a forward instead of a back slash. Python will convert the forward slash to the appropriate delimiter for the OS it is running on
open('C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Python 3.4\topheavy.txt','w')
OSError: [Errno 22] Invalid argument:'C:\ProgramData\Microsoft\Windows\Startmenu\Programs\Python 3.4\topheavy.txt'

Replace every \ with \\ in file path

My issue, rather arbitrary, was I was writing a file using open(filename, "w").write(...), where filename is an invalid pathname or includes unexpected slashes.
For example, converting a datetime.datetime.today() into a datestring with slashes or colons (Windows) and writing to non-existing directories will result in this error.
Change:
open("../backup/2021/08/03 15:02:61.json", "w").write(data)
To:
open("../backup/2021-08-03 15-02-61.json", "w").write(backup)
As an example.

Getting files in directory(directory has numbers in directory name)in python

I want to get files from directory (which has numbers in directory name). I am using below script. But it is throwing error.
yesterday=140402
os.chdir("C:\pythonPrograms\04-03-2014")
for file in glob.glob("MY*"+str(yesterday)+".log"):
print file
Error received:
WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\pythonPrograms\x04-03-2014'
Do I need to follow some convention while giving the path? The code works fine if I search in C:\pythonPrograms

"C:\pythonPrograms\04-03-2014"
The issue is the "\04", the \ character is used to denote an escape character, you may know about \n for new line. You can fix this by just doing:
os.chdir(r"C:\pythonPrograms\04-03-2014")
Which makes the string into a raw string. Or you can add another escape character to escape the escape character like:
"C:\\pythonPrograms\\04-03-2014"

Python os.chdir is modifying the passed directory name

I am trying to change the current working directory in python using os.chdir. I have the following code:
import os
os.chdir("C:\Users\Josh\Desktop\20130216")
However, when I run it, it seems to change the directory, as it comes out with the following error message:
Traceback (most recent call last):
File "C:\Users\Josh\Desktop\LapseBot 1.0\LapseBot.py", line 3, in <module>
os.chdir("C:\Users\Josh\Desktop\20130216")
WindowsError: [Error 2] The system cannot find the file specified
'C:\\Users\\Josh\\Desktop\x8130216'
Can anyone help me?

Python is interpreting the \2013 part of the path as the escape sequence \201, which maps to the character \x81, which is ü (and of course, C:\Users\Josh\Desktopü30216 doesn't exist).
Use a raw string, to make sure that Python doesn't try to interpret anything following a \ as an escape sequence.
os.chdir(r"C:\Users\Josh\Desktop\20130216")

You could also use os.path.join (documentation).
Example:
os.chdir(os.path.join('C:\Users\Josh\Desktop', '20130216'))
This is more elegant + it's compatible with different operating systems.

This should work -
os.chdir("C:\Users\Josh\Desktop\\20130216")

There are two to use os.chdir():
If you are using raw string than use single backslash \:
os.chdir(r"C:\Users\Josh\Desktop\20130216")
or
If you are not using raw string than use double backslash \\
os.chdir("C:\Users\Josh\Desktop\20130216")

I have faced the same problem but you have to try:
os.chdir(c:\\user\\Josh\\Desktop)
Use \\ so maybe you should get your solution.