After updating various packages (I'm running the Anaconda Python distribution) DataFrame.to_csv() will not work from within IPython Notebook. Everything else Pandas-related seems to work fine (reading data into datframes, manipulating them, etc.) but DataFrame.to_csv() will not generate an output file. A snippet of the code:
hives_output = hives_data.groupby(['user_id','replay']).apply(hives_processor).sort(['user_id','replay'])
hives_output.to_csv('hives_output.csv',index=False)
The first command runs fine, but no output file is produced by the second command. No error is produced whatsoever.
Weirder still, the code runs fine (i.e. it does generate an output file) within a regular Python or IPython interpreter, or when run from the command line, so the problem would seem to be specific to IPython Notebook.
I've tried downgrading various packages, but have yet to isolate the problem. Has anyone experienced this? Any ideas on what might be the cause? Some sort of incompatibility between IPython-Notebook and Pandas?
Related
I am trying to compile a .tex file in a Python script using pdflatex. The command pdflatex filename.tex works when I run it from the command line on Windows. However, attempting to run os.system("pdflatex filename.tex") just spits out 1 into the Python console and does not compile a pdf. I've also tried putting in the full file path similar to this person solved their problem but the same thing happens. Similarly, subprocess.call(['pdflatex', 'filename.tex']) just outputs a 1 and does not do anything.
It seems someone else has encountered the same problem in this thread, but on Mac instead of Windows. (But regardless of the operating system, they didn't find an answer.)
Why might this be happening?
EDIT: I've just discovered a solution. The script runs successfully (using the os.system approach) when I run the .py file using the command line. Previously I was attempting to run the script from RStudio, both using reticulate::source_python(filename) and also line-by-line via the reticulate REPL. Seems like the problem may actually be coming from R's reticulate package rather than anything to do with Python.
Fortunately RStudio has a terminal window so this doesn't end up being too inconvenient!
During a presentation yesterday I had a colleague run one of my scripts on a fresh installation of Python 3.8.1. It was able to create and write to a csv file in his folder (proof that the csv library was working correctly), but everything else failed due to not being able to find the needed files. To try and isolate the problem and figure out why, we tried the below simple script, which also failed.
He had this test.py script in "D:/TEST", which also contained some folders and image files. Running this script printed nothing to the console. No empty list, no error message, no newline. Maybe the print() function was also not working, but I didn't get around to testing that.
import os
print(os.listdir())
This script works fine on my computer and my other colleagues computers (all Windows 10, similar hardware). I didn't have time to look into the issue more thoroughly and don't have access to his computer anymore. What could be the problem? What other things could I have him look into in order to fix this? In case this problem appears again during a future presentation, what steps could I take to figure out the cause of it?
My colleague uninstalled Python and reinstalled it. After doing this apparently the "python" command will no longer run his scripts, but using "py" instead will. Now that he is using "py" to run his scripts, it is working as expected.
I started using Pycharm and managed to have the python prompt running after a file is run (link) and I found how to run pieces of code in the current iPython console (link).
For data science, however, it is very convenient to have a single iPython console/Python kernel where I can run files and code, just like Spyder does, and continue using the same started python kernel. For example to load a big amount of data with a script and write another script or pieces of code for plotting in different ways, exploring the data.
Is there a way in Pycharm to do it?
I think that it would imply generating automatically and running a line like:
runfile('C:/temp/my_project/src/console/load_data.py', wdir='C:/temp/my_project/src')
If the option does not exist, is it possible to make a macro or similar to do it?
The option "Single instance only" in Edit configurations doesn't help.
I am fairly new to programming in python. I installed anaconda and am running iPython (the Jupyter qtconsole) v.4.3.0 and python v.3.6 on a Mac. Currently, I am trying to import a module with functions located in my home directory.
I have looked at stackoverflow and python documentation and found that it could be done with:
%run "Users/myUser/python_functions.py"
or
import python_functions
However, when I try both of these approaches, I get prompted to overwrite the file that I am running or importing:
File `python_functions.py` exists. Overwrite (y/[N])?
This is changing the previous file and not getting the functions I want to be imported.
What may explain this, and what can I do to import my module?
this is wrong but leaving it up for shame
import on ubuntu (and I'm guessing many other unix-like OSs including Mac) is a utility that saves any visible window on an X server and outputs it as an image file. You can capture a single window, the entire screen, or any rectangular portion of the screen.
My guess if you are running the import command in your console, and it's about to take a screenshot and save it over an existing file - python_functions
Before you the use the python import command, start a python interpreter:
$ python
>>>import yourfile
edit: on re-reading your question, I'm not so sure about my guess anymore, but leaving it up until you tell me I'm wrong :)
Running Jupyter qtconsole as an interpreter is likely causing the problem in this scenario. Instead using a IDE or command line interpreter will resolve it .
Since anaconda was installed, trying it with the IDE Spyder executes the code just fine without the overwrite prompt. It works on others (e.g PyCharm, Rodeo, etc.) as well.
so I'm trying to familiarize myself with the Jupyter Notebook and am running into issues.
When I run the following code in the normal .py file of my PyCharm IDE it runs perfect; however, if I run it in my notebook the [*] never disappears meaning it just continuously runs never ending. Any idea why that might be the case? all answers much appreciated!
import pandas as pd
train_file='C:\Users\DDautel\Anaconda2\PycharmProjects\Kaggle\Titanic\RUN.csv'
test_file='C:\Users\DDautel\Anaconda2\PycharmProjects\Kaggle\Titanic\RUN2.csv'
train=pd.read_csv(train_file)
test=pd.read_csv(test_file)
print train.describe()
print test.describe()
For others gratification:
Glad it's solved.
Make sure python, ipython, jupyter from anaconda (using conda env's possibly) is in your path before others. Order is important.