I am running a python shell program in AWS Glue but after running for around 10 minutes its failing with error Internal service error. The logs or error logs does not give any information. Most of the time it fails by just saying Internal service error and rarely it runs for 2 days and gets timed out. The code uses pandas for transformations and it looks ok, it runs fine on local machine, necessary changes done so that it works on AWS glue[where it read/write files to s3 location instead of local folder]. What could be wrong here? Any input is appreciated.
This issue was figured out. The problem was the job was unable to download the dependent python libraries due to an access issue to the s3 bucket. Once the access issue was resolved the job started running fine.
Related
I'm trying to execute this jar file https://github.com/RMLio/rmlmapper-java from Airflow, but for some reason it is failing straight away. I'm using a PythonOperator to execute some python code, and inside it I have a subprocess call to the java command.
Test command is:
java -jar /root/airflow/dags/rmlmapper-6.0.0-r363-all.jar -v
I'm running Airflow inside a Docker container. The weird thing is that if I execute the exact same command inside the container it works fine.
I tried a bit of everything but the result is always the same: SegFault 139
The memory of the container seems to be fine so it shouldn't be directly related to some OOM issue. I also tried to reset default memory in the Docker compose file with no success.
My suggestion is that the java application somehow tries to load some files which are stored locally inside the jar file, but for some reason maybe Airflow changes the 'user.dir' directory and therefore it is not able to find them and it fails.
I'm really out of ideas so any help will be highly appreciated. Thank you.
had a problem when I was trying to deploy my django app with EB. I got something like this:
Instance has not sent any data since launch
each time I change options etc. The AWS in refers me to check my eb-engine.log file in which there is one error line:
[ERROR] An error occurred during execution of command [app-deploy] - [StageApplication]. Stop running the command. Error: staging application failed due to invalid zip file `
Moreover I consistently see in my 'Health section' app's environment 'No data'.
I've uploaded file with .zip extension as it is written. I checked my configuration file before as well as tried to change instance (to have more memory) and nothing worked. I'm pretty new in AWS and don't have really a clue how I can deal with that.
I have successfully created a py script that monitors network bandwidth and records it in a database. When I run it in debug everything works perfectly. However, when I start the service it errors with a 1053. If I comment out the import speedtest the windows service starts just fine. I know the issue is directly related to the module.
I searched the web to it's furthest reaches trying to figure this one out. I have increased my windows service timeout and made sure that my cx_freeze includes the speedtest module, among other small adjustments for testing purposes. All of which still result in the same error on windows service start.
Any help is much appreciated.
I have a linux function app deployed from a docker container on a premium app service plan running some python functions. One of my functions takes c.45 minutes to run as it is a bulk job. However, around 15 minutes into execution, the execution stops. I had a look in the logs in App Insights, and it always seems to stop with the following message and then has to restart the host:"AI: Local storage access has resulted in an error". This issue does not occur when I run locally.
Does anyone know what may cause this issue?
I am new to mrjob and I am having problems to get the job running on Amazon EMR. I will write them in sequential order.
I can run a mrjob on my local machine. However when I have mrjob.conf in /home/ankit/.mrjob.conf and in /etc/mrjob.conf, the job is not executed on my local machine.
Here is what I am getting. https://s3-ap-southeast-1.amazonaws.com/imagna.sample/local.txt
What is MRJOB_CONF in "the location specified by MR_CONF" in the documentation?
What is the use of 'base_tmp_directory' ? Also, do I need to upload the input data in S3 before starting the job or it will load from my local computer while starting the execution?
Do I need to do some bootstrapping if I use some libraries like numpy, scikit etc? If yes, how?
This is what I am getting when I execute the command for running a job on EMR https://s3-ap-southeast-1.amazonaws.com/imagna.sample/emr.txt
Any solutions?
Thanks a lot.
Your URL is invalid (I get an "Access Denied" error).
mrjob.conf is a configuration file. It can be located in several locations, see http://pythonhosted.org/mrjob/configs-conf.html
You can use input data from your local machine just by specifying the paths to the input files on the command line. MRJob will upload the data to S3 for you. If you specify an s3://... URL, MRJob will use the data at that S3 path.
To use non-standard packages, see http://pythonhosted.org/mrjob/writing-and-running.html#custom-python-packages
Your URL is invalid (I get an "Access Denied" error).