I'm using IronPython 2.7.7 for .Net to run some python code (Installed using NuGet in an MVC project).
I got a simple test case to work quite easily, but when I went to include the actual python code I want to call I ran into problems with missing modules.
No module named Crypto.Cipher Description: An unhandled exception
occurred during the execution of the current web request. Please
review the stack trace for more information about the error and where
it originated in the code.
Exception Details: IronPython.Runtime.Exceptions.ImportException: No module named Crypto.Cipher
These are the modules / libraries that the python script has at the beginning, I can see in visual studio that it's complaining about missing Crypto.Cipter and random.
from datetime import datetime
from Crypto.Cipher import AES
import time
import random
import socket
This is the .Net code that I'm calling the Python code with.
var basePath = System.Web.Hosting.HostingEnvironment.ApplicationPhysicalPath;
var realPath = Path.Combine(basePath, #"Python/PythonThingy.py");
var ipy = Python.CreateRuntime();
dynamic pyThing = ipy.UseFile(realPath);
var items = pyThing.discover(timeout: 5);
I am not allowed to install Python on the production server, so I'm hoping I can include these libraries in some way and avoid rewriting a quite huge set of functions. Any tips on how to do this?
P.S. If not apparent from post, I know very little about python.
Related
I have been trying to run a python script in NiFi's ExecuteScript processor. Though the catch here is that I don't have server file location access and all the python libraries are installed at "/data/jython", "/data/jython/Lib/site-packages/" and "data/nltk"
Below is the import section of my python script:
import json, traceback, pycountry, requests, geocoder, re, sys, nltk
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from nltk.corpus import stopwords
from java.nio.charset import StandardCharsets
from org.apache.commons.io import IOUtils
from org.apache.nifi.processor.io import StreamCallback
from org.python.core.util import StringUtil
I have added path reference to the packages/libraries:
Heres the screenshot of the error message:
Is there something I am missing? I have referred to another answer here, but couldn't figure out whats wrong with my code.
As the other answers state, Apache NiFi's ExecuteScript processor uses Jython, not Python. There is a limitation on the Jython library that it cannot handle native modules (modules that end in .so or are compiled C code, etc.). It is very likely that the pycountry module contains some native module. You can try a work-around proposed by Matt Burgess on the NiFi Developers Mailing List here.
ExecuteScript processor uses its own Jython Engine to execute your python scripts.
As the libraries which you are importing are not available in NIFI inbuild Jython Engine its throwing error.
SOLUTION:
If python is already installed on our machine with all those libraries (the same machine where your NIFI is installed) you can use that python engine
to execute your script. you can execute your python code using ExecuteProcess processor. see the configuration of ExecuteProcess.
This is a bit long so bear with me.
I am trying to learn both Python and Linux and am very new to both. I am currently doing some reading on deep learning from the following:
http://neuralnetworksanddeeplearning.com/chap1.html
I am attempting to import the mnist_loader package to use the associated data for testing the script that was previously written. However, upon typing import mnist_loader into the Linux command line, I was given the following:
"the program 'import' can be found in the following packages:"
at which point it listed some packages. Because I'm new to Linux and I don't have admin privileges, I decided to go a route that I understood better; that is to create a new python script and simply use the import command within (which has worked in all previous attempts).
I created a python script and tried import mnist_loader and received the following error:
"ModuleNotFoundError: No module named 'mnist_loader'"
I then checked my C drive and found that the file was indeed there. Here is a link to the Git repository where the files may be found:
https://github.com/MichalDanielDobrzanski/DeepLearningPython35
Next I moved on to trying to directly input the path to the file as follows:
import importlib.util
mnist_loader = importlib.util.spec_from_file_location("mnist_loader",r"C:\Users\XXXXXX\Documents\neural-networks-and-deep-learning-master\neural-networks-and-deep-learning-master\src\mnist_loader.py")
training_data, validation_data, test_data = mnist_loader.load_data_wrapper()
However, this produced the following error:
"AttributeError: 'ModuleSpec' object has no attribute 'load_data_wrapper'"
Note: the last line is used to collect the necessary data from the nist files.
I am running out of thoughts at this point and would love some feedback on all my "wrongdoings" up till now.
Thanks in advance!
P.S. It is worth noting that the book uses a package designed for Python 2.X whereas I am using 3.6. The readme provided by the book file location mentioned a different location where a Python 3.6 version could be found which is what I am going with.
Been a while since I have worked with Python, but I have some ideas as to what would cause the specific errors you are seeing. First I would suggest setting a PYTHON_PATH environment variable with the path you have that contains the module you want to import, this does not require administrator privileges fortunately. As for the load_data_wrapper attribute, you might have to do a from from mnist_loader import * to import all the functions inside the mnist_loader module.
I'm trying to use a Logo Language Compiler that uses Ply into the Unity3D environment for an Open Source project https://github.com/ssouzawallace/blocks-programming.
To do so I am using IronPython that is a Python interpreter running in .NET (I need this to run in Uinty3D). There is a bug in IronPython and i found others with the same issue related to the traceback of python script execution.
In resume if I run the Logo Compiler using the official Python interpreter everything goes OK. But in IronPython, when the code pass trough the get_caller_module_dic method it cannot find my pyLex stuff because it cannot reach the second frame level.
In order to resolve the problem I am wondering to pass the proper object or module to the method:
def lex(module=None,object=None,debug=0,optimize=0,lextab="lextab",reflags=0,nowarn=0,outputdir="", debuglog=None, errorlog=None):
But I don't know how to do this.
Someone know what can I do?
Thank you very much in advance
Found solution here Python: How do I get a reference to a module inside the module itself?
Now I pass the entire script as a module instead hoping the lex and yacc script find my module using the traceback.
Using
import sys
current_module = sys.modules[__name__]
import sys
import os
import logging
# need to add environment to apache's path for includes
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../")))
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../")))
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), ".")))
# likewise add cherrypy/other modules
sys.path.append(os.path.abspath("/Library/Python/2.7/site-packages/"))
import requests
response = requests.get('http://www.google.com').text
Using Python 2.7.6, requests 2.7.0, and Apache under MAMP 3.0, the above code crashes. A quick look through the code using winpdb seems to suggest that actually trying to open an internet connection is what is crashing the Python process. The Apache log is not very helpful, only saying
[notice] child pid 18879 exit signal Segmentation fault (11)
While my full code uses Cherrypy 3.8 to provide the WSGI portion of the framework, I feel that it is irrelevant to the problem at hand.
Is this some known problem with requests+apache, or is it some other problem? Python crashing without any comments makes it hard for me to even think of a way to start solving this issue.
EDIT: Using pdb, I found that the program segfaults on line 1421 of urllib.py in the python standard library.
proxy_settings = _get_proxy_settings()
where _get_proxy_settings comes from _scproxy.
I still have no idea how to fix this.
It seems that this is a platform-specific bug for my version of MacOS.
Im trying to import a company module into my software and I get the error:
ImportError: No module named config
from:
from pylons.config import config
So obviously, the module that im importing requires pylons.config but cant find it in my virtual environment.
If I go to the terminal and try some Python scripts I can seem to find the config file if I try:
from pylons import config
but will error if I try:
import pylons.config
Why is this?
And does anybody how or where I can get:
from pylons.config import config
to work. Bearing in mind that I cannot change the code for this module, only mine which is importing it or my own system files.
UPDATE
If anyone finding this page has a similar problem you may find that you are trying to run two modules with different versions of Pylons.
For example, you are creating a login application called myApp. You have some Python modules which help with login handling called pyLogin.
First you install pyLogin with python setup.py install. This adds the libraries to your site packages and updates any libraries it depends on, such as SqlAlchemy.
Next you install myApp in the same way which again updates libraries and dependencies.
This problem will occur if pyLogin and myApp are using different versions of Pylons. If pyLogin is using Pylons 0.9.6 and myApp is using Pylons 1.0 for example, then the pyLogin code will be called from myApp but it will be running in the wrong Pylons framework and hence will require EITHER from pylons import config or from pylons.config import config, but will only work with one. If it is using the wrong call for Pylons then you will find yourself with this error message.
So the only solution to this error is to either find earlier or later libraries which use the same Pylons version as your application or to convert your application to the same Pylons version as the libraries you are using.
There is a diffrence between two usages...
import loads a Python module into its own namespace, while from loads a Python module into the current namespace.
So, using from pylons import config imports config to to your current namespace. But trying to import a class or function using import is not possible since there is no namespace to keep them... You can only import modules, and use functions or classes via calling them with their own namespace like
import pylons
....
pylons.config #to retreive config
More about import in Python