I have a web application which uses Flask and talks to a database at the backend. It also uses Amazon AWS S3. I have written unit tests for this package.
The question is that I want to write integration tests where I am testing the external dependencies as well. I have been reading about integration tests and system tests. Do I create a new package lets says FooSystemTests or FooIntegrationTests or should they be part of my application package? I am planning to make this as part of my deployment process. My plan was in the integration tests I will test my external dependencies and in the system tests I will test things like if I go to a route what do I get back (testing as if the system is a black box). Also I read about selenium testing, where should that be system or integration?
Any thoughts/ideas will be very helpful.
Related
I'm building a website with React and Firebase that utilizes an algorithm I wrote in python. The database and authentication for the project are both handled by Firebase, so I would like to keep the cloud functions in that same ecosystem if possible.
Right now, I'm using the python-shell npm package to send and receive data from NodeJS to my python script.
I have local unit testing set up so I can test the https.onCall functions locally without needing to deploy and test from the client.
When I am testing locally, everything works perfectly.
However, when I push the functions to the cloud and trigger the function from the client, the logs in the Firebase console show that the python script is missing dependencies.
What is the best way to ensure that the script has all the dependencies available to it up on the server?
I have tried:
-Copying the actual dependency folders from my library/.../site-packages and putting them in the same directory under the /functions folder with the python script. This almost works. I just run into an issue with numpy: "No module named 'numpy.core._multiarray_umath'" is printed to the logs in Firebase.
I apologize if this is an obvious answer. I'm new to Python, and the solutions I've found online seem way to elaborate or involve hosting the python code in another ecosystem (like AWS or Heroku). I am especially hesitant to go to all that work because it runs fine locally. If I can just find a way to send the dependencies up with the script I'm good to go.
Please let me know if you need any more information.
the logs in the Firebase console show that the python script is missing dependencies.
That's because the nodejs runtime targeted by the Firebase CLI doesn't have everything you need to run python programs.
If you need to run a function that's primarily written in python, you should not use the Firebase CLI and instead uses the Google Cloud tools to target the python runtime, which should do everything you want. Yes, it might be extra work for you to learn new tools, and you will not be able to use the Firebase CLI, but it will be the right way to run python in Cloud Functions.
I'm deploying my Azure Function app using a CI/CD pipeline in Azure DevOps. The function invokes three console applications that are included in the package. One of the console applications is a stand alone .exe, it works without issue. The other two have dependencies to a number of dll:s that are also included in the package. This setup works well on my local machine, and when deployed using WebDeploy.
When instead deploying using Run From Package to a freshly created Function App Service, the function app itself loads fine as well as the standalone .exe console app, but both console apps that have dll dependencies fail to run, and both return with exit code 0xC0000135 to my function app (indicating that a dll failed to load).
Now, if I deploy once using Webdeploy and then deploy again using Run From Package, I get the latest build installed - and the console apps now work (!). I think this might be due to the .exe not being able to access the virtual file system when loading the dll:s, is this correct?
I could stick with WebDeploy but I really want to use the package deploy since the cold start time is much faster during scale-out (will need 100+ instances in production). I am also concerned that this way, the app actually needs to copy both the zip package, and the site structure under wwwroot, causing additional overhead.
What is the best way to include dependencies such as dll:s in a package when using Run From Package with Azure Functions?
(The function app is v3, built using .NET Core 3.1)
I am building a complex Python application that distributes data between very different services, devices, and APIs. Obviously, there is a lot of private authentication information. I am handling it by passing it with environmental variables within a Supervisor process using the environment= keyword in the configuration file.
I have also a test that checks whether all API authentication information is set up correctly and whether the external APIs are available. Currently I am using Nosetest as test runner.
Is there a way to run the tests in the Supervisor context without brute force parsing the supervisor configuration file within my test runner?
I decided to use Python Celery which is already installed on my machine. My API queries are wrapped as tasks and send to Celery. Given this setup I created my testrunner as just another task that runs the API tests.
The web application tests do not need the stored credentials but run fine in the Celery context as well.
Is there a common/best practice way to build a client-side javascript test platform for GAE python?
I know the GAE SDK has a Testbed API, and there's nose-gae. Both of these seem to be designed for unit-testing of server side code. I'm currently running on django-nonrel, and using django's unit test framework for this.
I'm looking for a similar testing framework where I can load fixture data, but run a real HTTP server so I can run Selenium tests against the server. This way I can test my javascript AJAX calls against fixed data.
I'm currrently achieving this by having django's test framework launch the GAE SDK dev_appserver on a separate thread after loading fixtures. This is able to handle HTTP calls from a browser. It mostly works for me, but there's a few cases where there's some flaky threading issues. The worst part is that it's rather hacky and calls deep into the SDK to launch dev_appserver. This is going to break with dev_appserver2.
I'm wondering if there's a better way to solve this. With all the GAE devs out there, someone else must be running system tests against fixed data?
Has anyone done tests on production (or on staging) as opposed to local tests on the dev server? is it a bad idea to try?
At first glance, App Engine and unit tests aren't a great fit. App
Engine requests can only be driven by http or xmpp. Unit tests are
typically initiated via command-line or IDE. App Engine requests are
limited to 30 seconds. A unit test suite may contain thousands of
tests that take far longer than 30 seconds to execute. How do we
bridge the gap?
is there a python equivalent to cloud cover?
I would love my app to have a webpage of checkbox's which allows me to select which tests to run and displays recent results for each test. (preferably without me writing my own version of unittest / unittest2
While some of my tests may be local only, I think I may need to run some of these tests on production also. I may also have additional "live only tests".
I guess my concern is how to run the local tests on live without having to maintain two separate sets of tests. And also how to run some tests on live without messing up the live data in the datastore. (yes some tests may use stubs or mocks but I might wanna check the production datastore or a staged version of it?)
I haven't tried running a unit test on live, I assume via stdout unittest would log results to the administration console which probably wouldn't be as useful as having the results show up on a webpage that is used for running the tests.
I would also want to set up staging before production by changing the version number in app.yaml (combined with Namespaces, Versions and Multitenancy.. ). I could run tests on staging also.
Anyone got a basic approach I should try?
Check out aeta. It runs tests in the task queue, and you can access these tests from the web interface or the command line.
For testing the live datastore without messing up your data, you could try using a staging or testing server.
Have you tried the remote_api console? It will allow you to run unit tests inside your local directory straight onto the live appengine runtime.