I have a rest API (Python based). The underlying logic calls an Oracle procedure that refreshes certain materialized views. The application is hosted on Openshift Container Platform. Now sometimes the app gets stuck on this step( refresh materialized views).
Is there a way to add a liveness probe here that restarts the container if the app remains stuck at this step for some amount of time say 2 hours.
Is there a way to add a liveness probe here that restarts the container if the app remains stuck at this step for some amount of time say 2 hours.
Yes, that would be possible, however you would need to implement the logic yourself.
Liveness probes typically check the return code of a command or the HTTP response from a REST endpoint. So in your case you would likely need to create a new REST endpoint that checks and will return an error if any step is taking longer than a certain time. If that is the case, the endpoint should return a HTTP error code such as 500.
Related
We have Azure http triggered function app(f1) which talks to another http triggered function app(f2) that has a prediction algorithm.
Depending upon input request size from function(f1), the response time of function(f2) increase a lot.
When the response time of function(f2) is more, the functions get timed out at 320 seconds.
Our requirement is to provide prediction algorithm as a
service(f2)
An orchestration API(f1) which will be called by the client and
based on the clients input request (f1) will collect the
data from database do data-validation and pass the data to
(f2) for prediction
After prediction (f2) would respond back predicted result to
(f1)
Once (f1) receives the response from (f2), (f1) would respond
back to client.
We are searching for alternative azure approach or solution which will
reduce the latency of an API and also the condition is to have f2
as a service.
If it takes more than 5 minutes in total to validate user input, retrieve additional data, feed it to the model and run the model itself, you might want to look at something different than APIs that return response synchronously.
With these kinds of running times, I would recommend a asynchronous pattern, such as F1 stores all data on a Azure Queue, F2 (Queue triggered) runs the model and stores data in database. Requestor can monitor database for updates. Of F1 takes the most time, than create a F0 that stores the request on a Queue and make F1 queue triggered as well.
As described in Limits for Http Trigger:
If a function that uses the HTTP trigger doesn't complete within 230 seconds, the Azure Load Balancer will time out and return an HTTP 502 error. The function will continue running but will be unable to return an HTTP response.
So it's not possible to make f1 and/or f2 Http Triggered.
Alternatives are many, but none can be synchronous (due to limitation above) if:
Interface to end user is REST API and
API is served by Http Triggered Azure Function and
Time needed to serve request is greater than 230 seconds.
Assuming:
Interface to end user is REST API and
API is served by Http Triggered Azure Function
one async possibility would be like this:
PS: I retained f1 and f2, which do the same as in your design. Though their trigger/output change.
Http Triggered REST API from f3 is the entry point for end user to trigger the job. Which would post to queue q1 and return a job-id / status-url as response.
user can query/poll current status/result of job by job-id using another Http Trigger API served by f4.
f1 and f2 are now triggered by queue trigger
f1, f2 and f3 update status for each job-id whenever needed into ADLS (which could be anything else like Redis cache or Table Storage etc).
f4 need not a separate function, it can be served as a different path/method from f3.
I want to get info from API via Python, which has infinite info updating (it is live - for example live video or live monitoring). So I want to stop this GET request after interval (for example 1 second), then process these information and then repeat this cycle.
Any ideas? (now I am using requests module, but I do not know, how to stop receiving data and then get them)
I might be off here, but if you hit an endpoint at a specific time, it should return the JSON at that particular moment. You could then store it and use it in whatever process you have created.
If you want to hit it again, you would just use requests to hit the endpoint.
So I'm trying to get data from an API which has a max call limit of 60/min. If I go over this it will return a response [429] too many requests.
I thought maybe a better way was to keep requesting the API until I get a response [200] but I am unsure how to do this.
import requests
r = requests.get("https://api.steampowered.com/IDOTA2Match_570/GetTopLiveGame/v1/?key=" + key + "&partner=0")
livematches = json.loads(r.text)['game_list']
So usually it runs but if it returns anything other than a response [200], my program will fail and anyone who uses the website won't see anything.
Generally with cases like this, I would suggest adding a layer of cache indirection here. You definitely don't want (and can't) try solving this using frontend like you suggested since that's not what the frontend of your app is meant for. Sure, you can add the "wait" ability, but all someone has to do is pull up Chrome Developer tools to grab your API key and then call that as many times as they want. Think of it like this: say you have a chef that can only cook 10 things per hour. Someone can easily come into the restaurant, order 10 things, and then nobody else can order anything for the next hour, which isn't fair. Instead, adding a "cache" layer, which means that you only call the steam API every couple seconds. If 5 people request your site within, say, 5 seconds, you only go to the steam API on the first call, then you save that response. To the other 4 (and anyone who comes within those next few seconds), you return a "cached" version.
The two main reasons for adding a cache API layer are the following:
You would stop exposing key from the frontend. You never want to expose your API key to the frontend directly like this since anyone could just take your key and run many requests and then bam, your site is now down (denial of service attack is trivial). You would instead have the users hit your custom mysite.io/getLatestData endpoint which doesn't need the key since that would be securely stored in your backend.
You won't run into the rate limiting issue. Essentially if your cache only hits the API once every minute, you'll not run into any time where users can't access your site due to an API limit, since it'll return cached results to the user.
This may be a bit tricky to visualize, so here's how this works:
You write a little API in your favorite server-side language. Let's say NodeJS. There are lots of resources for learning the basics of ExpressJS. You'd write an endpoint like /getTopGame that is attached to a cache like redis. If there's a cached entry in the redis cache, return that to the user. If not, go to the steam API and get the latest data. Store that with an expiration of, say, 5 seconds. Boom, you're done!
It seems a bit daunting at first but as you said being a college student, this is a great way to learn a lot about how backend development works.
I have a Cloud Function that does some searching on my database. It's quite an expensive process that involves a lot of computing. Its goal is to find the best possible match. I have no way doing an early exit when a good match is found because a better one might be found later in the iteration cycle. So ideally I want to update the client regularly about the best matches so far. The issue is for a HTTP Cloud Function I can only return once. I was thinking of updating the results on the database somewhere and have the client listen in for changes, however, this may not be much faster since there will be some delay in that process too. So is there a way of returning multiples responses to a HTTP query in a Cloud Function? Is there a better solution to this I am not seeing?
Pseudo of what I need
def cloudFunction(someData):
goodMatches = []
for i in database:
if (i == goodMatch):
goodMatches.append(goodMatch)
post new goodMatch to client
return goodMatches
An HTTP type function can only have one response, and it will be sent in its entirety. Cloud Functions does not support HTTP chunking or streaming of results. If you want to send progressive results, consider writing those into Cloud Firestore or Realtime Database at a unique location that's agreed upon between the client and the function.
I'm using django to develop a website. On the server side, I need to transfer some data that must be processed on the second server (on a different machine). I then need a way to retrieve the processed data. I figured that the simplest would be to send back to the Django server a POST request, that would then be handled on a view dedicated for that job.
But I would like to add some minimum security to this process: When I transfer the data to the other machine, I want to join a randomly generated token to it. When I get the processed data back, I expect to also get back the same token, otherwise the request is ignored.
My problem is the following: How do I store the generated token on the Django server?
I could use a global variable, but I had the impression browsing here and there on the web, that global variables should not be used for safety reason (not that I understand why really).
I could store the token on disk/database, but it seems to be an unjustified waste of performance (even if in practice it would probably not change much).
Is there third solution, or a canonical way to do such a thing using Django?
You can store your token in django cache, it will be faster from database or disk storage in most of the cases.
Another approach is to use redis.
You can also calculate your token:
save some random token in settings of both servers
calculate token based on current timestamp rounded to 10 seconds, for example using:
token = hashlib.sha1(secret_token)
token.update(str(rounded_timestamp))
token = token.hexdigest()
if token generated on remote server when POSTing request match token generated on local server, when getting response, request is valid and can be processed.
The simple obvious solution would be to store the token in your database. Other possible solutions are Redis or something similar. Finally, you can have a look at distributed async tasks queues like Celery...