When trying to display a dataframe with a column called 'cmdline', using Jupyter Notebook, the value is being truncated and after a google search I found the following solution:
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', None)
pd.set_option('display.max_colwidth', -1)
df1.head(n = 15)
but none of this helped. The 'cmdline' column is still too narrow and not showing the full text in the column. Does anyone know of a solution. The above is all I could find using a Google search.
edit:
the dataframe was generated by
import sqlite3
conn = sqlite3.connect("ram-usage.db")
df1 = pd.read_sql("select datetime, cmdline, sum(resident) as resident, sum(shared) as shared from processdata where datetime > '2019-09-25' and datetime < '2019-09-26' group by datetime, cmdline ;", conn)
The version of software components I copy and pasted below:
Server Information:
You are using Jupyter notebook.
The version of the notebook server is: 5.7.0
The server is running on this version of Python:
Python 3.6.6 | packaged by conda-forge | (default, Oct 12 2018, 14:08:43)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
Current Kernel Information:
Python 3.6.6 | packaged by conda-forge | (default, Oct 12 2018, 14:08:43)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.0.1 -- An enhanced Interactive Python. Type '?' for help.
The notebook itself is run from Docker
docker run --group-add=root --rm -p 8888:8888 -e "PASSWORD=xxxxxxxx" -e "USE_HTTP=1" -v "$PWD":/home/jovyan jupyter/scipy-notebook
if you want to look at the database file itself it can be downloaded from http://files.kiwiheretic.xyz/ram-usage.tar.gz or http://files.kiwiheretic.xyz/ram-usage.db
I will suggest you run your code in a different IDE (like Spyder). Then you can view the entire dataframe by simply clicking on the object in the variable explorer.
Related
I noticed this issue when trying to run pip commands on my WSL2 Ubuntu-20.04 machine that it would take a long time to run anything. I finally narrowed it down to when pip is importing the keyring module.
Python 3.8.5 (default, May 27 2021, 13:30:53)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> def import_keyring():
... time_start = time.time()
... import keyring
... print(f"Keyring took {time.time() - time_start} seconds to load")
...
>>> import_keyring()
Keyring took 400.4930064678192 seconds to load
looking at the process explorer, it seems there is a child process that it is waiting for:
dbus-launch --autolaunch <32 character hex string> --binary-syntax --close-stderr
the dbus-launch command itself does not seem to have any child processes, so I used strace to see where it's hanging:
connect(3, {sa_family=AF_INET, sin_port=htons(6000), sin_addr=inet_addr("<my main machine's IP>")}, 16) = 0
It looks to be trying to connect to the host windows computer on port 6000 and failing? Now I'm just at a loss of what is happening and why it's taking so long to fail. Any help appreciated!
I had this ready to ask when finally clicked on what port 6000 meant. I tried setting up X server on Windows for WSL2 to use a long time ago.
Either get the X server working or unset your DISPLAY environment variable and this should clear it right up.
export DISPLAY=
Hope this helps anyone else running into this obscure issues!
I'm trying to connect to a remote python interpreter but I'm getting the following error in the Python Console:
ssh://ubuntu#131.175.21.168:22/usr/bin/python3.6 -u /home/ubuntu/.pycharm_helpers/pydev/pydevconsole.py 0 0
Couldn't connect to console process.
Process finished with exit code -1
I'm using the following:
Pycharm version: Pycharm Professional 2017.3.4.
Project interpreter: Remote Python 3.6.5 (ssh connection to a remote virtual ubuntu 16.04 server built thru OpenStack).
Deployment: SFTP, auth type: Key pair with private key file.
The SFTP connection works fine, synchronization of the code to the remote server also works fine. But I can't seem to run any code in the Python Console.
Any ideas on how to solve this?
Thanks in advance for the help!
When I try to connect directly to port 22, I get an error like below.
ssh://keiichi.kuroyanagi#mygcp:22/home/keiichi.kuroyanagi/.pyenv/shims/python -u /home/keiichi.kuroyanagi/.pycharm_helpers/pydev/pydevconsole.py 0 0
Couldn't connect to console process.
Process finished with exit code -1
I now connect to remote console by configuring portforwarding in RoyalTSX as below.
The console is as follows.
ssh://keiichi.kuroyanagi#localhost:10022/home/keiichi.kuroyanagi/.pyenv/shims/python -u /home/keiichi.kuroyanagi/.pycharm_helpers/pydev/pydevconsole.py 0 0
import sys; print('Python %s on %s' % (sys.version, sys.platform))
sys.path.extend(['/home/keiichi.kuroyanagi/tmp/pycharm_project_444', '/home/keiichi.kuroyanagi/tmp/pycharm_project_444'])
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.1.1 -- An enhanced Interactive Python. Type '?' for help.
PyDev console: using IPython 7.1.1
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44)
[GCC 7.3.0] on linux
In[2]:
The following link may also be helpful.
Can't connect to remote interpreter python console : PY-18029
https://youtrack.jetbrains.com/issue/PY-18029
My pycharm version is as follows.
PyCharm 2018.2.4 (Professional Edition)
Build #PY-182.4505.26, built on September 19, 2018
Licensed to DeNA Co.,Ltd. / Keiichi Kuroyanagi
You have a perpetual fallback license for this version
Subscription is active until October 24, 2019
JRE: 1.8.0_152-release-1248-b8 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.13.6
Using a Python script, I'd like to get the email of a person who last committed changes to a specific file /path/to/file.py.
Sounds easy, right? I just need to somehow parse the following
git log -n 1 --pretty=format:%ae -- /path/to/file.py
Package sh is my preferred choice. Unfortunately, in Python
import sh
print(str(sh.git.log('-n 1 --pretty=format:%ae -- /path/to/file.py')))
print(str(sh.git.log('-n', '1', '--pretty=format:%ae', '--', /path/to/file.py')))
both print - (press RETURN).
So maybe I'm messing something up with the arguments.
Otherwise, str(sh.git.status()) correctly returns On branch master ..., and some other tested commands work as expected.
How to solve this?
The - (press RETURN) output sounds like it's something printed by a pager.
Remember, every Git command may (depending on options, arguments, configuration settings, and other environmental details such as whether stdin is a tty) run its output through a pager. The pager used depends on your personal configuration. How that pager acts depends on the pager used and on the input data.
One simple workaround is to run git --no-pager <git-command> to tell Git not to use a pager, even if the configuration and environment suggest that Git should use a pager.
This should work:
print(str(sh.git.log("-n 1", "--pretty=format:%ae", "/path/to/file")))
At least this shows how it works on my machine:
$ git log -n 1 --pretty=format:%ae -- README.md
foo#bar.com
$ python3
Python 3.6.4 (default, Jan 25 2018, 15:54:40)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sh
>>> print(str(sh.git.log("-n 1", "--pretty=format:%ae", "README.md")))
foo#bar.com
Im trying to do a simple Ansible script to attach a volume in EC2
- ec2_vol:
instance: XXXXXX
volume_size: 5
device_name: sdd
When running I get the following error
msg": "Region eu-central-1 does not seem to be available for aws
module boto.ec2. If the region definitely exists, you may need to
upgrade boto or extend with endpoints_path"
When I check that eu-central-1 is accesible via python boto... it is there:
$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto.ec2
>>> boto.ec2.regions()
[RegionInfo:us-west-1, RegionInfo:us-east-1, RegionInfo:ap-northeast-1, RegionInfo:ap-southeast-2, RegionInfo:sa-east-1, RegionInfo:ap-northeast-2, RegionInfo:us-east-2, RegionInfo:ap-southeast-1, RegionInfo:ca-central-1, RegionInfo:cn-north-1, RegionInfo:us-west-2, RegionInfo:us-gov-west-1, RegionInfo:ap-south-1, RegionInfo:eu-central-1, RegionInfo:eu-west-1, RegionInfo:eu-west-2]
Here are the versions of my tools
ansible 2.3.2.0
aws-cli 1.11.151
Python 2.7.12
Linux 4.4.0-93-generic
botocore 1.7.9
boto 2.48.0
pip 9.0.1
I already check most of the things I can think of and even reproduce it on another virtual machine with vagrant and it gives me the same error..what else can I check?
Do you run this with connection: local? – Konstantin Suvorov 6 hours
ago
Answer from the comments:
Do you run this with connection: local?
Cloud modules like ec2_vol are (usually) supposed to be executed from localhost (where all required libraries and credentials are stored).
Importing cx_Oracle in a python script fails.
I have cx_Oracle installed, using "pip install cx_oracle" - that worked fine, reported installed.
Now when i try:
import cx_Oracle
I get the following error
Traceback (most recent call last):
File "reader.py", line 9, in <module>
import cx_Oracle
ImportError: dlopen(/Library/Python/2.7/site-packages/cx_Oracle.so, 2): Symbol not found: _OCIAttrGet
Referenced from: /Library/Python/2.7/site-packages/cx_Oracle.so
Expected in: flat namespace
in /Library/Python/2.7/site-packages/cx_Oracle.so
Other Information:
Python version 2.7 / mac os 10.7.2 (Lion)
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Oracle 10.2
$ sqlplus -version
SQL*Plus: Release 10.2.0.4.0 - Production
Also, I do not have a /bin directory at all in my ORACLE_HOME folder, I have only the instant client and SDK installed.
ox_Oracle
$ pip freeze
PyRSS2Gen==1.0.0
...
cx-Oracle==5.1.1
(found a lot of questions on getting cx_Oracle installed, but none on this - thanks)
I ran into this problem today and was able to solve it by changing path to the libraries referenced in the InstantClient binaries to the actual locations on the filesystem.
This blog http://blog.caseylucas.com/2013/03/03/oracle-sqlplus-and-instant-client-on-mac-osx-without-dyld_library_path/ provides detailed explanation and the script to adjust all binaries. The only problem is that it uses #executable_path , which does not seem to work anymore with Python 2.7 & El Capitan (I'm not really sure what is responsible for the security exception). Replacing #executable_path with the actual path works just fine.
To summarize, steps to make it work:
install InstantClient to /usr/local/instantclient_11_2
make sure that cx_Oracle.so shared object that you use is at /Library/Python/2.7/site-packages/cx_Oracle.so
copy the following script to /usr/local/instantclient_11_2
#!/bin/sh
# script to change the dynamic lib paths and ids for oracle instant client
# exes and libs
(echo /Library/Python/2.7/site-packages/cx_Oracle.so ; find . -maxdepth 1 -type f \( -perm -1 -o \( -perm -10 -o -perm -100 \) \) -print ) | while read exe
do
echo adjusting executable $exe
baseexe=`basename $exe`
otool -L $exe | awk '/oracle/ {print $1}' | while read lib
do
echo adjusting lib $lib
baselib=`basename $lib`
if [ "$baseexe" = "$baselib" ]
then
echo changing id to $baselib for $exe
install_name_tool -id $baselib $exe
else
echo changing path id for $lib in $exe
install_name_tool -change $lib /usr/local/instantclient_11_2/$baselib $exe
fi
done
done
run the script with root permissions.
Uninstall everything.
Then install oracle instant client:
http://digitalsanctum.com/2007/07/26/installing-oracle-instant-client-on-mac-os-x/
Then use pip to install cx_oracle.
http://www.iceycake.com/2012/02/tutorial-how-to-install-cx_oracle-python-on-mac-os-x-lion/
Then set the path to point to the 32 bit version of oracle.
edit .profile file in your home directory and add the path to your oracle bin home, using this line:
export PATH=$PATH:/usr/local/lib/instantclient/
And it works...