Develop Reference

Opening fits files with no SIMPLE keyword in python

I have a large number of fits files that I'm unable to open becuase of a missing SIMPLE keyword. When I try to open them using astropy's fits.open() it gives the following error:
OSError: No SIMPLE card found, this file does not appear to be a valid FITS file
I tried adding the ignore_missing_simple=True option to fits.open(), but this still gives an error:
OSError: Empty or corrupt FITS file
I'm not sure what to do here. My first thought was to edit the fits header to add a SIMPLE keyword, but if I can't open the file I don't know how to deal with this. I'm wondering if theres more going on than just the missing SIMPLE keyword given the second error.
If it matters, I've downloaded the fits files, I did not generate them myself.

We traced the problem to astropy.io.fits version 4.3
Files generated with version 4.2 can be read with 4.0 or 4.2 but this error pops up with 4.3.1. You could try again after downgrading astropy.io.fits.

Which version of astropy are you using? The problem occurs on 4.3, but it works just fine in 4.2.
This GitHub issue reports the problem in the astropy repository.
I propose you downgrade until the issue is fixed.

Trying to open shapefiles

I have a number of companion files for a shapefile - .cpg, .dbf, .prj, .sbn, .sbx, .shp, .rtree, .shx
I have tried downoading a number of free softwares that open shapefiles such as TatukGIS viewer and DBF Reader Plus but with no luck.
I've attempted to open them in Python as I read this article:
http://www.acgeospatial.co.uk/geopandas-shapefiles-jupyter/
I'm having trouble installing geopandas however. I get this error when I try to run:
I've looked into these errors and tried using upgrade setup tool
Totally lost, have no idea what this error means when I look it up

As to you original question, try QGIS software. Among lots of features it surely can open Shapefile as well as many other geospatial formats.
As to the error you've posted, it's related to user permissions and was already answered, e.g. here Access is denied when trying to pip install a package on Windows.

Apparently geopandas is notoriously difficult to install, especially on Windows. To install geopandas, first install Conda, open the Anaconda Prompt as administrator and type:
conda install -c conda-forge geopandas
Found the answer on here:
https://www.reddit.com/r/gis/comments/as5rze/what_does_this_geopandas_installation_error_mean/
You can then use geopandas to open shapefiles

Installing Python Pandas

I am currently using Linux Ubuntu 16.04 and python as the language. I am trying to convert an XLSX file to pdf and I came across this question with answer. Python - Converting XLSX to PDF.
One of the answer requires me to install pandas and also wkhtmltopdf and use them to convert the files into pdf format.
I tried installing pandas but after installing pandas, I am face with builtins.ImportError: No module named 'pandas'.
So far I have tried this solution but it did not work for me.
ImportError: No module named pandas
Is it possible for me not to also install Anaconda to help me.
Can someone guide me through this.
Thanks!

If you are using PyCharm IDE, you can try:
Settings => Project Interpreter => plus button => search package to install

Error writing to parquet file using pyspark

I am working on windows 10. I installed spark, and the goal is to use pyspark. I have made the following steps:
I have installed Python 3.7 with anaconda -- Python was added to C:\Python37
I download wintils from this link -- winutils is added to C:\winutils\bin
I downloaded spark -- spark was extracted is: C:\spark-3.0.0-preview2-bin-hadoop2.7
I downloaded Java 8 from AdoptOpenJDK
under system variables, I set following variables:
HADOOP_HOME : C:\winutils
SPARK_HOME: C:\spark-3.0.0-preview2-bin-hadoop2.7
JAVA_HOME: C:\PROGRA~1\AdoptOpenJDK\jdk-8.0.242.08-hotspot
And finally, under system path, I added:
%JAVA_HOME%\bin
%SPARK_HOME%\bin
%HADOOP_HOME%\bin
In the terminal:
So I would like to know why I am getting this warning:
unable to load native-hadoop library... And why I couldn't bind on port 4040...
Finally, inside Jupyter Notebook, I am getting the following error when trying to write into Parquet file. This image shows a working example, and the following one shows the code with errors:
And here is DataMaster__3.csv on my disk:
And the DaterMaster_par2222.parquet:
Any help is much appreciated!!

If you are writing the file in csv format, I have found that the best way to do that is using the following approach
LCL_POS.toPandas().to_csv(<path>)
There is another way to save it directly without converting to pandas but the issue is it ends up getting split into multiple files (with weird names so I tend to avoid those). If you are happy to split the file up, its much better to write a parquet file in my opinion.
LCL_POS.repartition(1).write.format("com.databricks.spark.csv").option("header", "true").save(<path>)
Hope that answers your question.

How to install something that read excel files in python 3.3

I am new to python and I need to read an excel file in pyscripter. I understand that I should
import xrld
in Python and then use:
book = xlrd.open_workbook(os.path.join(file_source,file_name +".xlsx"))
to open and read the excel file.
However, I just don't understand how to install this module 'xrld'. I have looked on several websites and even on stackoverflow itself, but I just don't understand anything from the explanations that are given. I just don't understand how to install 'easy install' or 'pip' (or understand what these things are in the first place) or understand how to install anything else I need to read an excel file for. So far I installed a file called 'setuptools-36.01.1' from https://pypi.python.org/pypi/setuptools which should contain something I could use for running, but I just don't understand what to do.
I thereby must admit (and think it is clear by now) that I am a real noob when it comes to programming or computers in the first place.
Can someone please help me by explaining in plain English what exactly to do in order to read an excel file in python. I have Python 3.3.2 and use Pyscripter to run my simulation.
Your help would be really appreciated!

EasyInstall (easy_install) gives you a quick and painless way to install packages remotely by connecting other websites via HTTP.
Pip is a package management system used to install and manage software packages, such as those found in the Python Package Index. Pip is a replacement for easy_install.
Since you have installed the setuptools, easy_install will be already present in your system.
So just run easy_install pip in the command line to install pip . Then pip install xlrd for installing xlrd.
example code:
import xlrd
workbook = xlrd.open_workbook('C:/User/myaccount/Book1.xls')
worksheet = workbook.sheet_by_name('Sheet1')
print worksheet
Please go through this link-Reading Excel file using Python XLRD for sample codes for reading data from excel files in python.
Hope this helps you..