I am trying to use OpenAI's API to play with some of the examples they have. However, when I go to load my API key, I get errors. I created a ".env" file and did:
OPENAI_API_KEY=XYZ-123
and then in Python I have the following:
import os
import openai
openai.api_key_path = ".env"
openai.api_key = os.getenv("OPENAI_API_KEY")
print(openai.Model.list())
Every time it tells me my API key is malformed. I can also remove the 3rd line and I get the same error that it is malformed but I copied it directly into the .env file from the website. Also, if I set the key directly in Python, it seems to work just fine:
openai.api_key = "XYZ-123"
But for security, I would prefer I don't see the key in my Python code. Any suggestions on how to resolve this?
Create a .properties document and add only your API key in this document without any quotation marks or anything. The API key should be the only text in the document. Pass the path for this document in for the value of openai.api_key_path and it should work.
Remember that the value expects the path from the root directory. If you make it in the root directory, just pass in ".properties". If you make it in a sub directory called backend, for example, pass in "backend/.properties".
Hope this helps.
I suggest using dotenv for this purpose:
pip install python-dotenv
Usage example:
import dotenv
config = dotenv.dotenv_values(".env")
openai.api_key = config['OPENAI_API_KEY']
It is pretty flexible and works well whenever storing secrets in .env files comes up. Don't forget to add them to .gitignore or use .env.local (ignored by default)!
Setting the openai.api_key_path did not seem to work for me. Once I deleted its value, my code started working.
Does not work
This probably searches the path first even if the api_key value is set, and then throws an error.
openai.api_key_path = '.env'
openai.api_key = os.getenv("OPENAI_API_KEY")
Works
Initialize the api_key_path to None again.
openai.api_key_path = None
openai.api_key = os.getenv("OPENAI_API_KEY")
I believe your .env file OpenAI key needs to be in the format:
OPENAI_API_KEY="XYZ-123"
Related
I want to get all the environment variables starting with a specific prefix and save them (without prefix) into a dictionary. Is there any better way than getting all the os.environ variables and searching through them?
I also need those to be merged with a config file, so if you know any library in python which is like Viper in go (which handles both environment variables and config files and merging of them with priority), it will be a huge help.
UPDATE:
my configs are not simple app config, they are users config with some structures in it, so it's not a simple key value pair. It also might be in different formats, such as YAML, INI, JSON, etc.
I don't know any other solution that using os.environ to load your environment variable and loop through them. It would look something like:
import os
import re
# Replace PREF by the prefix you want
prefix="PREF"
myPattern = re.compile(r'{prefix}\w+'.format(prefix=prefix))
my_env_variables = {key.replace(prefix,''):val for key, val in os.environ.items() if myPattern.match(key)}
print(my_env_variables)
Regarding config files management, I warmly recommend the python-dotenv library: documentation
Searching through os.environ is going to be the best way.
from typing import Dict
import os
prefix: str = "MY_PREFIX_"
variables: Dict[str, str] = {}
for key, value in os.environ.items():
if key[:len(prefix)] == prefix:
variables[key[len(prefix):]] = value
If you are reading the env vars from .env file or from terminal than environs package is the best
export KAFKA_HOST=kafka
export KAFKA_PORT=9092
Python code to read all the env variables starting with KAFKA_
from environs import Env
env = Env()
env.read_env()
with env.prefixed("KAFKA_"):
print(env("HOST"))
print(env("PORT"))
How can I get the path to the %APPDATA% directory in Python?
import os
print os.getenv('APPDATA')
You may use os.path.expandvars(path):
Return the argument with environment variables expanded. Substrings of the form $name or ${name} are replaced by the value of environment variable name. Malformed variable names and references to non-existing variables are left unchanged.
On Windows, %name% expansions are supported in addition to $name and ${name}.
This comes handy when combining the expanded value with other path components.
Example:
from os import path
sendto_dir = path.expandvars(r'%APPDATA%\Microsoft\Windows\SendTo')
dumps_dir = path.expandvars(r'%LOCALAPPDATA%\CrashDumps')
Although the question clearly asks about the Windows-specific %APPDATA% directory, perhaps you have ended up here looking for a cross-platform solution for getting the application data directory for the current user, which varies by OS.
As of Python 3.11, somewhat surprisingly, there is no built-in function to find this directory. However, there are third-party packages, the most popular of which seems to be appdirs, which provides functions to retrieve paths such as:
user data dir (user_data_dir)
user config dir (user_config_dir)
user cache dir (user_cache_dir)
site data dir (site_data_dir)
site config dir (site_config_dir)
user log dir (user_log_dir)
You can try doing:
import os
path = os.getenv('APPDATA')
array = os.listdir(path)
print array
You can use module called appdata. It was developed to get access to different paths for your application, including app data folder. Install it:
pip install appdata
And after that you can use it this way:
from appdata import AppDataPaths
app_paths = AppDataPaths()
app_paths.app_data_path # for your app data path
app_paths.logs_path # for logs folder path for your application
It allows to to get not only app data folder and logs folder but has other features to manage paths like managing config files paths. And it's customizable.
Links:
Read the Docs - documentation.
GitHub - source code.
PyPI - package manager (pip).
I printed this page and am gratfull for this.
Also tryed to configure "%APPDATA%"in this Dispositive,
using: "notepad", doesn't knowing to archive the sugested condiguration. Also copied sotoz and Aominé contribuiters. tks.
efk14it.
I am getting "Lambda can't find the file lambda_function.py. Make sure that your handler upholds the format: file-name.method." error just above the AWS Lambda function code block.
You can see the warning here
But as you can see in this code block, my file name is lambda_function and my function name is lambda_handler.
import boto3
import os
import logging
import uuid
from webdriver_screenshot import WebDriverScreenshot
logger = logging.getLogger()
logger.setLevel(logging.INFO)
s3 = boto3.client('s3')
def lambda_handler(event, context):
logger.info('## ENVIRONMENT VARIABLES')
logger.info(os.environ)
screenshot_file = "{}-{}".format(''.join(filter(str.isalpha, os.environ['URL'])), str(uuid.uuid4()))
driver = WebDriverScreenshot()
logger.info('Generate fixed height screenshot')
driver.save_screenshot(os.environ['URL'], '/tmp/{}-fixed.png'.format(screenshot_file), height=1024)
logger.info('Generate full height screenshot')
driver.save_screenshot(os.environ['URL'], '/tmp/{}-full.png'.format(screenshot_file))
driver.close()
if all (k in os.environ for k in ('BUCKET','DESTPATH')):
## Upload generated screenshot files to S3 bucket.
s3.upload_file('/tmp/{}-fixed.png'.format(screenshot_file),
os.environ['BUCKET'],
'{}/{}-fixed.png'.format(os.environ['DESTPATH'], screenshot_file))
s3.upload_file('/tmp/{}-full.png'.format(screenshot_file),
os.environ['BUCKET'],
'{}/{}-full.png'.format(os.environ['DESTPATH'], screenshot_file))
My code folder is like this and the main python code is in src folder. In AWS Lambda, my folder is looking like this. I do not understand what is the problem.
Even worse, when I try to test the code in AWS Lambda environment, this error is showing up. It says:
Unable to import module 'lambda_function': No module named 'boto3'
But I have the boto3 module in the layer, it is uploaded from my S3 Bucket. It is succesfully added to the function
The only thing I am suspecting, I may wrote environment variables wrong as shown here: environment variables due to I am a beginner in coding and AWS Lambda, I just copied both the PATH and PYTHONPATH from somewhere.
I don't know what to do next and need some support here.
Thanks.
I faced the same problem when I tried to use an external module in lambda_function, in my case was OpenCV as lambda layer. For making Opencv to work I have to change environment variables, making the key PYTHONPATH to have the value /opt/ as is describe it in this tutorial. Then I have the same problem you describe Unable to import module 'lambda_function': No module named 'boto3', what is kind of strange as boto3 suppose to be integrate in the path of lambda. In my understanding, is just a conflict with the directories.
The way it solves for me was changing the value of the environment variables from /opt/ to /opt:/var/runtime. Hopefully it helps you.
No worries, I can help you with this. For the first Lambda can't find the file lambda_function.py issue, try prepending src/ to the handler value in the function configuration since your Python scripts are not located in the top-level directory. For the second issue related to lambda layers, it's true that you messed up the PYTHONPATH environment variable. If you are using layers, you don't need to overwrite this variable. Just leave it unchanged.
We are working on an add-on that writes to a log file and we need to figure out where the default var/log directory is located (the value of the ${buildout:directory} variable).
Is there an easy way to accomplish this?
In the past I had a similar use case.
I solved it by declaring the path inside the zope.conf:
zope-conf-additional +=
<product-config pd.prenotazioni>
logfile ${buildout:directory}/var/log/prenotazioni.log
</product-config>
See the README of this product:
https://github.com/PloneGov-IT/pd.prenotazioni/
This zope configuration can then be interpreted with this code:
from App.config import getConfiguration
product_config = getattr(getConfiguration(), 'product_config', {})
config = product_config.get('pd.prenotazioni', {})
logfile = config.get('logfile')
See the full example
here: https://github.com/PloneGov-IT/pd.prenotazioni/blob/9a32dc6d2863b5bfb5843d441e652101406d9a2c/pd/prenotazioni/init.py#L17
Worth noting is the fact that the initial return avoids multiple logging if the init function is mistakenly called more than once.
Anyway, if you do not want to play with buildout and custom zope configuration, you may want to get the default event log location.
It is specified in the zope.conf. You should have something like this:
<eventlog>
level INFO
<logfile>
path /path/to/plone/var/log/instance.log
level INFO
</logfile>
</eventlog>
I was able to obtain the path with this code:
from App.config import getConfiguration
import os
eventlog = getConfiguration().eventlog
logpath = eventlog.handler_factories[0].instance.baseFilename
logfolder = os.path.split(logpath)[0]
Probably looking at in the App module code you will find a more straightforward way of getting this value.
Another possible (IMHO weaker) solution would be store (through buildout or your prefered method) the logfile path into an environment variable.
You could let buildout set it in parts/instance/etc/zope.conf in an environment variable:
[instance]
recipe = plone.recipe.zope2instance
environment-vars =
BUILDOUT_DIRECTORY ${buildout:directory}
Check it in Python code with:
import os
buildout_directory = os.environ.get('BUILDOUT_DIRECTORY', '')
By default you already have the INSTANCE_HOME environment variable, which might be enough.
From within an App Engine app, is there a way to determine the project ID a GAE (App Engine) instance is running on?
I want to access a big query table in the same project that the App Engine instance is running in. I'd rather not hard code it in or include it in another config file if possible.
Edit: forgot to mention that this is from Python
This is the "official" way:
from google.appengine.api import app_identity
GAE_APP_ID = app_identity.get_application_id()
See more here: https://developers.google.com/appengine/docs/python/appidentity/
You can get a lot of info from environment variables:
import os
print os.getenv('APPLICATION_ID')
print os.getenv('CURRENT_VERSION_ID')
print os.environ
I tried the other approaches in 2019 using Python3. So far as I can tell, those approaches are for Python2 (and one for Java).
I was able to accomplish this in Python3 using:
import os
app_id = os.getenv('GAE_APPLICATION')
print(app_id)
project_id = os.getenv('GOOGLE_CLOUD_PROJECT')
print(project_id)
source: https://cloud.google.com/appengine/docs/standard/python3/runtime
I also added an app version, in case you need it too.
import com.google.appengine.api.utils.SystemProperty;
String appId = SystemProperty.applicationId.get();
String appVersion = SystemProperty.applicationVersion.get();