Stagged validation/parsing in Amazon Lex - python

I want to make a simple quote bot which needs to know about addresses and dimensions. Currently I have hacked one together with a python state machine and rasa NLU for the intent stuff, though the nature of the data means most of the entities are better extracted by hand.
For example, an opening sentence might be "i want to send 4 boxes from A postcodeA to B postcodeB they're 3 by 3 by 3m each". These addresses need to be validated, which may involve a back and forth (did you mean this postcode, or maybe that state? This isn't a valid match, please choose from this list, etc...).
Additionally, the order of the validation might be weird. For example, one could have a suburb, postcode, and is_valid slot. There are several possibilities: they could enter a valid S + P, just S, an invalid S alone, a valid S and P which are individually valid but not a match, etc. In some cases I want to defer validation to make use of other information, for example if an invalid suburb was given but a valid postcode, i would use the postcode to inform suggestions for the correct suburb. However, if an invalid suburb was given but no postcode, there is no point asking for a postcode because i would want to get a valid suburb first. In this way having a slot ordering of 'suburb', 'postcode', 'is_valid' doesn't quite cut it as far as i can tell.
I suppose the validation call for suburb and postcode can just have their own set of switch statements which look at the other slots which may have been filled at the same time? Seems a bit chicken and egg, as would need to wait for validation of those as well, and may end up requiring calling Lambda's from within Lambda's. For example, if the validation fails, the bot asks for more info, but the validation of that info may require a different process/lambda than the original input
It could also be they don't include any details about the object, in which case it needs to be prompted for later after the address stuff is all sorted out. For example, if they fail to include info about dimensions i want to ask 'what're the dimensions' and allow them to either include just measurements or weights and quantities as well - this i understand as standard slot filling.
So my question is: how difficult/reasonable is it to have validation calls in Lambdas actually lead to side-paths in the conversation? Previously i had done something like this in a slot-filling setting by having 'is_valid' slots all over the place (especially if I don't want to simply throw an 'isn't valid' error and robotically re-ask the original question'). Is there a better way?
Also, how would one manage 'interrupt' intents? That is, a set of intents which would trigger 'do you want to go restart?' kind of questions, which will return to the original state if the user replies 'no', and, ideally, if they say yes can return to a specific point in the conversation (which i imagine would be achieved by just resetting appropriate slots to empty)
Also, not wedded to the idea of Amazon Lex, any of the pre-canned cases which reduce amount of boiler plate code are good too.
Apologies for word vomit.

Wow, ok here's what I can offer:
1. Validation Difficulty
how difficult/reasonable is it to have validation calls in Lambdas actually lead to side-paths in the conversation?
Building a complex validation and conversation algorithm is standard practice for Lex Developers of service bots. So it is very reasonable to expect to have to do that yourself either in Lambda, or somewhere else and just use Lambda as a go-between. So the difficulty lies in your hands, and maybe you can find APIs that you can use to validate addresses and postal codes like Google Maps to make that part easier.
2. 'is_valid' Slots
Previously i had done something like this in a slot-filling setting by having 'is_valid' slots all over the place (especially if I don't want to simply throw an 'isn't valid' error and robotically re-ask the original question'). Is there a better way?
Lex does have a better way: sessionAttributes! A good practice is to only create slots for holding the values you need to fulfill each intent. For anything else, you can happily rely on sessionAttributes to keep track of the conversation path, slot validity, slot history, intent history, interruption intents, etc, etc, etc, as much as you can imagine. It's completely up to you how to organize your bot logic and keep track of current and past states of the convo there.
For example: You could have slot: postalCodeA
And in sessionAttributes also have: postalCodeA_valid, postalCodeA_confirmed, postalCodeA_attempts etc.
And use those sessionAttributes values to determine the conversation path in your logic. When you find a slot is invalid, you could save that value in an ..._attempts or ..._history list in sessionAttributes, then set ..._valid to false, and reset the slot to null, and re-elicit that slot with a message explaining why it was invalid, or try to elicit an address slot instead of a postal code slot.
3. Interrupt Intents
Also, how would one manage 'interrupt' intents? That is, a set of intents which would trigger 'do you want to go restart?' kind of questions
As I hinted at earlier, the answer to this is also sessionAttributes! When your user is inside of one intent (intentA), Lex will first attempt to fill the current elicit slot with their input, but if it doesn't match, Lex will also check if the input matches a different intent's utterances. So you could have an interrupt intent (intentB) with utterances like "let's start over", "nevermind", "back up", etc. Then in all of your normal intents, you keep a backup of that intent's slot values in sessionAttribtues, as well as something like last_intent to know where a user was previously in case it changes.
This would allow you to handle interrupt intents like this:
user enters intentA
intentA fills some slots, and backs them up in sessionAttributes
user says "lets start over" triggering intentB
intentB asks for confirmation of canceling intentA
(Yes) intentB fulfills the intent after erasing intentA values from sessionAttributes and returns user to start with elicitIntent and asks "how else may I help you?"
(No) intentB passes the user back to intentA (which you know because you kept track of sessionAttributes.last_intent) and send a confirmation of continuing with intentA with confirmIntent: "Alright, I still remember where we left off, would you like to continue {intentA action}?" (the response will be sent to intentA, where you can handle that).
(if user wants to continue with intentA) intentA fills the new empty slots from sessionAttributes and uses other sessionAttributes values to continue through your algorithm to the point where it left off, delivering the same elicit slot that it was last on, and the user is impressed with your bot's intelligence. =)

Related

How to trigger a "undo" for user input

I have the following user input statements that are part of a larger function. When prompted for user input it is the case that a user might fat finger the keyboard and continue with the following input prompts. The user input is arbitrary and it really depends on them if they plug in the correct information. However, it could be neat to go back to the previous input statement and continue from there by triggering a keybind (E.g crtl + b lets say) or something similar.
I understand that this could be refined by having explicit invalid inputs with a conditional statement. however, this wouldn't be the use case for my situation. I'm not sure how to go about this. Hoping someone can point me in the right direction.
whichDIMM = input("\nWhich DIMM needs to be replaced?: (ex. A4) \n")
dimmSize = input("\n64G/32G/16G memory?\n")
hostname = input("\nEnter hostname: \n"
Would it be possible to 'Undo' and go back to the previous prompt in case there are spelling mistakes? then continue as expected? otherwise, the user will need to re-run the entire program.
For example:
Which DIMM needs to be replaced?
*mistypes*
64G/32G/16G memory?
(trigger undo)
Which DIMM needs to be replaced?
DIMM_A4
64G/32G/16G memory?
64G
Enter hostname:
myhostname
...
continue with the rest of the script
You are basically trying to turn this into a form where you can move between fields and then submit the whole form. This has been written in many ways already so you could just use one such library. A simple one would be npyscreen for example.

Developing Amazon Alexa skill and I am having trouble with slot values (acronyms) being read as words

I am building a TV guide as an Amazon Alexa skill and have all of the necessary parts working (I.E. Lambda, DynamoDB, and Alexa Skill console) where the majority of channels will be recognized by voice, found in my database, and returned.
The reason for this question is I want Alexa to take my voice input of "BET", but every time I spell out BET, she doesn't recognize the slot. If I say "bet" she will recognize the slot, uppercase the slot (part of my lambda function), match to word in my database, and return the item. I would prefer to spell out the word rather than just say "bet".
How do I ensure that when I spell out "BET" that Alexa will understand "BET" and not "bet"?
Make sure you are properly tokenizing the abbreviations in your slots in addition to your utterances, this will allow Alexa to match what you are saying to the values:
https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/alexa-skills-kit-voice-interface-and-user-experience-testing#writing-conventions-for-sample-utterances
"BET" would become "B. E. T."
You may have to adjust your Lambda code to expect these new values (perhaps all it would take is a REGEX replace on the slot value).
--
Edit: For channels like FOX, it might make sense to include both "FOX" and "F. O. X." (as well as any common spoken alternatives) as slot values and process them on the Lambda side.

How to process multi-line text in Spark

Suppose I have a dictionary with words and explanations in the following format:
Firefox
<web> A complete {free}, {open-source} {web
browser} from the {Mozilla Foundation} and therefore a true
code descendent of {Netscape Navigator}. The first non-{beta
release} was in late 2004.
{Firefox Home (http://mozilla.org/products/firefox)}.
(2005-01-26)
firehose syndrome
<networking, jargon> An absence, failure or inadequacy of flow
control mechanisms causing the sender to overwhelm the
receiver. The implication is that, like trying to drink from
a firehose, the consequenses are worse than just loss of data,
e.g. the receiver may {crash}.
See {ping-flood}.
[{Jargon File}]
(2007-03-12)
firewall
1. {firewall code}.
2. {firewall machine}.
firewall code
1. The code you put in a system (say, a telephone switch) to
make sure that the users can't do any damage. Since users
always want to be able to do everything but never want to
suffer for any mistakes, the construction of a firewall is a
question not only of defensive coding but also of interface
presentation, so that users don't even get curious about those
corners of a system where they can burn themselves.
2. Any sanity check inserted to catch a {can't happen} error.
Wise programmers often change code to fix a bug twice: once to
fix the bug, and once to insert a firewall which would have
arrested the bug before it did quite as much damage.
[{Jargon File}]
How would I map this type of data so that I can do things with it in spark?

Making sure only one order (or less) is executing for a single player at a given time

To best describe the question, i'll begin with the following scenario:
Suppose i have a poker game:
The player is allowed to use credit in order to purchase some goods.
If the player executes two purchase orders at the same time (theoretically), two workers may handle this request simultaneously and there could be an integrity error, thus the application must make sure that there is only one (or less) orders executing for a single player at a given time.
Just to make sure that the scenario is clear - there could be hundreds of orders executing simultaneously - but for different players
Following the 12Factor guidelines, i should be able to scale out the workers (which actually process the purchase orders).
How can i make sure that only one order (or less) is executing for a single player at a given time with an elegant solution?
Thanks in advance,
Erik.
Just a short disclaimer:
I'm no python expert nor a database expert but I fell that my solution below is on the right track. Apply it with your skills and you'll get the result youre looking for.
It's one of a few ways of achieving what you'd like dare I say quite elegant :).
Im assuming you have different devices hence how multiple transactions could take place for one particular customer.
Based on that assumption and also the assumption that you have some sort of webservice for the system, I have solved it like this:
Add a table in your database that will record all active customers being handled currently.
Then you add one more check before a transaction with a customer begins on the device to make sure that customer is not on that active transactions table, if so, then reject the new transaction from happening, otherwise continue. This check before hand will ensure that only one transaction can happen for any particular customer at any time no matter how many devices are being used concurrently.
The above is just an example. Its how I am currently handling it, and its working well for me.
It's basically the idea of quickly checking whether that current customer is in the activecustomers database, and if so for you to simply reject the customer as that customer is already in a session elsewhere. Once the customer is done with that session, you delete the entry from that table.
Our current solution involves a RabbitMQ exclusive queue (every consume locks the queue until the task is finished) for each player and celery to consume tasks from the queues.
What do you think of this proposed solution?
Erik.

Is there a security risk when using a user-inputted variable in sqlite3.connect()?

I am having users both create and connect to a database through a python application,
dbname = raw_input("Please enter the game name you wish to connect to, or the name of a new game: ")
db = sqlite3.connect(dbname)
I'm curious: is there any kind of input a potentially malicious user could use to attack the database (providing I have no other vulnerabilities in the other handlers), or to attack the underlying system running python? (I am aware that a user could continue to create an infinite number of databases by running the program over and over, however I can't think of any other solution than quotas for that).
Thanks in advance!
Travis
They can pass in ":memory", which may or may not have an adverse effect on the system, depending on what your application does.
They can control transactions (via the isolation_level parameter), which might mess up your program's interaction with the database.
I would check user-input against a dictionary of possible database names.
You shouldn't allow that. Imagine what could happen if the user enters "Global Thermonuclear War"?
In all seriousness, you'll probably want to give the user a menu with IDs for example 1..5 associated with the games, then in your code use something like a switch case statement to establish the DB name to connect to.
Usually the connections themselves are not something to worry about. It's any queries that can be malicious or destructive. Having a user enter the name of the database to connect to poses no security threat that I can see, but make sure yourqueries are risk-free from injection.
When in doubt sanitize the data because:
garbage in == garbage out
if user inputs garbage the program will most likely output garbage
To quote:
All input is, until proven otherwise, (1) malicous, (2) willing to
break your code, and (3) totally capable of doing so. (That said, this
doesn't look like the typical security loophole.) – delnan
...
Sanitize it by escaping, encoding, etc. Then check against a whitelist, not a blacklist (because it does not check for unknown dangers unlike a whitelist)

Categories

Resources