Minimize Base64 - python

So, I have a code in C# that converts the Image to base64 and vice versa. Now, I want to send the generated base64 to python.
Here's my existing code.
var startProcess = new ProcessStartInfo
{
FileName = pythonInterpreter,
Arguments = string.Format($"\"{pythonPathAndCode}\" {b64stringCSharp}"),
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardInput = true,
RedirectStandardError = true,
CreateNoWindow = true,
};
using (Process process = Process.Start(startProcess))
{
error = process.StandardError.ReadToEnd();
testResult = process.StandardOutput.ReadToEnd();
lblTestOutput.Text = testResult;
lblError.Text = error;
}
This code is working just fine when I'm trying to send a small value of string to python. But when it comes in sending base64 value, an exception error appeared.
System.ComponentModel.Win32Exception: 'The filename or extension is too long'
Take note that the code is working perfectly fine when I'm sending only 32,000 string and less but base64 consist of exactly 98,260.
Is there a way to minimize this base64?
This is my python code:
import sys
inputFromC = sys.stdin
print("Python Recevied: ", inputFromC)

The maximum length for a command + arguments in Windows is 32767 characters (link). This is consistent with that you're seeing.
I recommend sending the image over the process's standard input instead. Something like:
var startProcess = new ProcessStartInfo
{
FileName = pythonInterpreter,
Arguments = string.Format($"\"{pythonPathAndCode}\""),
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardInput = true,
RedirectStandardError = true,
CreateNoWindow = true,
};
using (Process process = Process.Start(startProcess))
{
process.StandardInput.Write(b64stringCSharp);
process.StandardInput.Close();
error = process.StandardError.ReadToEnd();
testResult = process.StandardOutput.ReadToEnd();
lblTestOutput.Text = testResult;
lblError.Text = error;
}
Obviously, modify your Python script to read from standard input, rather than a command-line argument.

Related

Getting output from Python files when executed through C#

Im calling Python script from a C# app to execute with the code below
string srCommandName = "customWhisper.py D:\Tas\Monitor\Stemme_226.m4a 41f850e7-455e-4f84-b1eb-a5cccea49046.txt"
ProcessStartInfo psInfo = new ProcessStartInfo(srCommandName);
psInfo.UseShellExecute= false;
psInfo.RedirectStandardOutput = true;
using (Process process = Process.Start(psInfo))
{
using (StreamReader reader = process.StandardOutput)
{
string result = reader.ReadToEnd();
if (result=="")
{ }
}
}
My Python code is:
try:
if len(sys.argv)==3:
mediaPath = sys.argv[1]
textFilePath = sys.argv[2]
model = whisper.load_model("tiny")
isExist = os.path.exists(mediaPath)
if isExist:
results = model.transcribe(mediaPath, language = "da")
stab_segments = results['segments']
text_file = open(textFilePath, "w",encoding="utf-8")
for i in stab_segments:
text_file.write(i["text"] + '\n')
text_file.close()
print(0)
else:
print(1)
else:
print(len(sys.argv))
except Exception as er:
print(er)
The desired output from string result = reader.ReadToEnd(); should had been 0 (print 0) on success or one of the other prints. But its the srCommandName
Is there any way to get Python to return a value when called from C#
I use the below method to call python script from c# and get the result from python standard output back as a string. The FileName property of ProcessStartInfo should point to the python interpreter to use - not the python script. The python script should be sent as first argument.
private (string output, int exitCode) RunPythonScript(string pathToPythonFile, string args)
{
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = #"C:\MyPathToPythonInterpreter\Scripts\python.exe";
start.Arguments = string.Format("{0} {1}", pathToPythonFile, args);
start.UseShellExecute = false;
start.RedirectStandardOutput = true;
using (Process process = Process.Start(start))
{
process.WaitForExit();
using (StreamReader reader = process.StandardOutput)
{
string result = reader.ReadToEnd();
return (result, process.ExitCode);
}
}
}

inititialize_mint instruction failed: invalid account data for instruction

first and foremost, I presume that initialize_mint is the process to create a token.
I have this program written that seems to me to be ok without an issue:
use solana_program::{
account_info::{next_account_info, AccountInfo},
entrypoint,
entrypoint::ProgramResult,
pubkey::Pubkey,
program::{invoke},
};
use spl_token::{instruction::*};
entrypoint!(process_instructions);
pub fn process_instructions(program_id: &Pubkey, accounts: &[AccountInfo], _: &[u8]) -> ProgramResult{
let account_info_iter = &mut accounts.iter();
let mint_account_info = next_account_info(account_info_iter)?;
let rent_account_info = next_account_info(account_info_iter)?;
let mint_authority_pubkey = program_id.clone();
let freeze_authority_pubkey = program_id.clone();
let decimals = 0;
let token_program_info = next_account_info(account_info_iter)?;
invoke(
&initialize_mint(&token_program_info.key, &mint_account_info.key, &mint_authority_pubkey, Some(&freeze_authority_pubkey), decimals)?,
&[mint_account_info.clone(), rent_account_info.clone(), token_program_info.clone()]
)?;
Ok(())
}
But then when I proceed to call it from the client, I am using this code: edited to include create_account as pointed out by #Jon C
payer_account_meta = AccountMeta(payer_keypair.public_key, True, True)
spl_program_meta = AccountMeta(TOKEN_PROGRAM_ID, False, False)
rent_account_meta = AccountMeta(solana.sysvar.SYSVAR_RENT_PUBKEY, False, False)
mint_keypair = Keypair.generate()
mint_account_meta = AccountMeta(mint_keypair.public_key, False, True)
create_account_params = system_program.CreateAccountParams(payer_account_meta.pubkey, mint_account_meta.pubkey, Token.get_min_balance_rent_for_exempt_for_mint(client), 82, spl_program_meta.pubkey)
client.send_transaction(Transaction().add(
system_program.create_account(create_account_params)
), payer_keypair, mint_keypair)
accounts = [
mint_account_meta,
rent_account_meta,
spl_program_meta
]
transaction = Transaction()
transaction.add(TransactionInstruction(
accounts,
program_id,
bytes([])
))
client.send_transaction(transaction, payer_keypair)
running this give me the error:even after including create_account it still gives thesame error
What is the account data I am getting wrong please?
Very close! You need to create an account before initializing it. You have two options:
create the account in a separate instruction, as seen in the solana-py client: https://github.com/michaelhly/solana-py/blob/f41f020938d1fb257142f18608bcb884adb54479/src/spl/token/core.py#L114
create the account from within your program, using a cross-program invocation, similar to https://solanacookbook.com/references/programs.html#create-a-program-derived-address but using invoke instead of invoke_signed, and without the program-derived address / seeds
Note that this all requires a signature from the mint keypair.
Edit post modification:
It looks like you're still generating a new keypair in mint_account_meta instead of using mint_keypair, so instead of Keypair.generate(), try:
mint_account_meta = AccountMeta(mint_keypair.public_key, False, True)

Sending and rendering multiple images from Flask backend to Flutter frontend

I've been trying to send multiple images from my flask server to flutter. I've tried everything, I either get a byte cannot be json serialised or flutter gives error in parsing the image. I've been using Image.memory() for the response.
The weird part is, if I send over one image in bytes format, it works as intended.
Any help is greatly appreciated
#app.route('/image', methods = ['POST'])
def hola():
with open("1.jpg", "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
return encoded_string
This server side code works as intended. Following is the code I used for Flutter
Future<String> uploadImage(filename, url) async {
// List<String> images;
var request = http.MultipartRequest('POST', Uri.parse(url));
request.files.add(
await http.MultipartFile.fromPath('picture', filename),
);
request.headers.putIfAbsent('Connection', () => "Keep-Alive, keep-alive");
request.headers.putIfAbsent('max-age', () => '100000');
print(request.headers.entries);
http.Response response =
await http.Response.fromStream(await request.send());
print("Result: ${response.statusCode}");
// print(y);
return response.body;
// return res;
}
Then I call this function with help of an on button click event. like this:
var res = await uploadImage(file.path, url);
setState(() {
images = res;
});
Container(
child: state == ""
? Text('No Image Selected')
: Image.memory(base64.decode(images)),
),
The above is the working example it renders the Image I send. The following is where I face problem:
Server Side:
#app.route('/send', methods= ['GET'])
def send():
with open("1.jpg", "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
with open("2.jpg", "rb") as image_file:
encoded_string2 = base64.b64encode(image_file.read())
x = [str(encoded_string2), str(encoded_string)]
return jsonify({'images':x})
To handle the above here is my flutter code:
var request = http.MultipartRequest('POST', Uri.parse(url));
request.files.add(
await http.MultipartFile.fromPath('picture', filename),
);
request.headers.putIfAbsent('Connection', () => "Keep-Alive, keep-alive");
request.headers.putIfAbsent('max-age', () => '100000');
print(request.headers.entries);
http.Response response =
await http.Response.fromStream(await request.send());
print("Result: ${response.statusCode}");
var x = jsonDecode(response.body);
var y = x['images'];
var z = y[0];
images = z;
To render the image the container code remains the same. I get this error:
The following _Exception was thrown resolving an image codec:
Exception: Invalid image data
or I get:
Unexpected character at _
I tried parsing in a different manner, for ex:
var x = jsonDecode(response.body);
var y = x['images'];
var z = utf8.encode(y[0]);
images = base64Encode(x[0]);
or this:
var x = jsonDecode(response.body);
var y = x['images'];
var z = base64Decode(y[0]);
images = z;
but nothing works
if you are trying to return several image binaries in a response I assume looks something like
{ "image1":"content of image one bytes", "image2":"content of image two bytes" }
and as you have found a problem arises in that binary content cannot be encoded naively into json
what you would typically do is convert it to base64
{"image1":base64.b64encode(open("my.png","rb").read()),"image2":open(...)}
most things can render base64 (not entirely sure specifically for flutter Images (but certainly for tags in html)
<img src="data:image/png;base64,<BASE64 encoded data>" />
If not you can always get the bytes back with base64decode (which flutter almost certainly has in one of its libraries)

Hex to string, the python way, in powershell

A bit of a weird question perhaps, but I'm trying to replicate a python example where they are creating a HMAC SHA256 hash from a series of parameters.
I've run into a problem where I'm supposed to translate an api key in hex to ascii and use it as secret, but I just can't get the output to be the same as python.
>>> import hmac
>>> import hashlib
>>> apiKey = "76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b"
>>> apiKey.decode('hex')
'v\xb0,\x0cEC\xa8^EU$fiL\xf6w\x93x3\xc9\xcc\xe8\x7f\nb\x82H\xaf-,I['
If I've understood the material online this is supposed to represent the hex string in ascii characters.
Now to the powershell script:
$apikey = '76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b';
$hexstring = ""
for($i=0; $i -lt $apikey.Length;$i=$i+2){
$hexelement = [string]$apikey[$i] + [string]$apikey[$i+1]
$hexstring += [CHAR][BYTE][CONVERT]::toint16($hexelement,16)
}
That outputs the following:
v°,♀EC¨^EU$fiLöw?x3ÉÌè⌂
b?H¯-,I[
They are almost the same, but not quite and using them as secret in the HMAC generates different results. Any ideas?
Stating the obvious: The key in this example is no the real key.
Update:
They look more or less the same, but the encoding of the output is different. I also verified the hex to ASCII in multiple online functions and the powershell version seems right.
Does anyone have an idea how to compare the two different outputs?
Update 2:
I converted each character to integer and both Python and Powershell generates the same numbers, aka the content should be the same.
Attaching the scripts
Powershell:
Function generateToken {
Param($apikey, $url, $httpMethod, $queryparameters=$false, $postData=$false)
#$timestamp = [int]((Get-Date -UFormat %s).Replace(",", "."))
$timestamp = "1446128942"
$datastring = $httpMethod + $url
if($queryparameters){ $datastring += $queryparameters }
$datastring += $timestamp
if($postData){ $datastring += $postData }
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$apiAscii = HexToString -hexstring $apiKey
$hmacsha.key = [Text.Encoding]::ASCII.GetBytes($apiAscii)
$signature = $hmacsha.ComputeHash([Text.Encoding]::ASCII.GetBytes($datastring))
$signature
}
Function HexToString {
Param($hexstring)
$asciistring = ""
for($i=0; $i -lt $hexstring.Length;$i=$i+2){
$hexelement = [string]$hexstring[$i] + [string]$hexstring[$i+1]
$asciistring += [CHAR][BYTE][CONVERT]::toint16($hexelement,16)
}
$asciistring
}
Function TokenToHex {
Param([array]$Token)
$hexhash = ""
Foreach($element in $Token){
$hexhash += '{0:x}' -f $element
}
$hexhash
}
$apiEndpoint = "http://test.control.llnw.com/traffic-reporting-api/v1"
#what you see in Control on Edit My Profile page#
$apikey = '76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b';
$queryParameters = "shortname=bulkget&service=http&reportDuration=day&startDate=2012-01-01"
$postData = "{param1: 123, param2: 456}"
$token = generateToken -uri $apiEndpoint -httpMethod "GET" -queryparameters $queryParameters, postData=postData, -apiKey $apiKey
TokenToHex -Token $token
Python:
import hashlib
import hmac
import time
try: import simplejson as json
except ImportError: import json
class HMACSample:
def generateSecurityToken(self, url, httpMethod, apiKey, queryParameters=None, postData=None):
#timestamp = str(int(round(time.time()*1000)))
timestamp = "1446128942"
datastring = httpMethod + url
if queryParameters != None : datastring += queryParameters
datastring += timestamp
if postData != None : datastring += postData
token = hmac.new(apiKey.decode('hex'), msg=datastring, digestmod=hashlib.sha256).hexdigest()
return token
if __name__ == '__main__':
apiEndpoint = "http://test.control.llnw.com/traffic-reporting-api/v1"
#what you see in Control on Edit My Profile page#
apiKey = "76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b";
queryParameters = "shortname=bulkget&service=http&reportDuration=day&startDate=2012-01-01"
postData = "{param1: 123, param2: 456}"
tool = HMACSample()
hmac = tool.generateSecurityToken(url=apiEndpoint, httpMethod="GET", queryParameters=queryParameters, postData=postData, apiKey=apiKey)
print json.dumps(hmac, indent=4)
apiKey with "test" instead of the converted hex to ASCII string outputs the same value which made me suspect that the conversion was the problem. Now I'm not sure what to believe anymore.
/Patrik
ASCII encoding support characters from this code point range 0–127. Any character outside this range, encoded with byte 63, which correspond to ?, in case you decode byte array back to string. So, with your code, you ruin your key by applying ASCII encoding to it. But if what you want is a byte array, then why do you do Hex String -> ASCII String -> Byte Array instead of just Hex String -> Byte Array?
Here is PowerShell code, which generate same results, as your Python code:
function GenerateToken {
param($apikey, $url, $httpMethod, $queryparameters, $postData)
$datastring = -join #(
$httpMethod
$url
$queryparameters
#[DateTimeOffset]::Now.ToUnixTimeSeconds()
1446128942
$postData
)
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.Key = #($apikey -split '(?<=\G..)(?=.)'|ForEach-Object {[byte]::Parse($_,'HexNumber')})
[BitConverter]::ToString($hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($datastring))).Replace('-','').ToLower()
}
$apiEndpoint = "http://test.control.llnw.com/traffic-reporting-api/v1"
#what you see in Control on Edit My Profile page#
$apikey = '76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b';
$queryParameters = "shortname=bulkget&service=http&reportDuration=day&startDate=2012-01-01"
$postData = "{param1: 123, param2: 456}"
GenerateToken -url $apiEndpoint -httpMethod "GET" -queryparameters $queryParameters -postData $postData -apiKey $apiKey
I also fix some other errors in your PowerShell code. In particular, arguments to GenerateToken function call. Also, I change ASCII to UTF8 for $datastring encoding. UTF8 yields exactly same bytes if all characters are in ASCII range, so it does not matter in you case. But if you want to use characters out of ASCII range in $datastring, than you should choose same encoding, as you use in Python, or you will not get the same results.

Can Python read from a Windows Powershell namedpipe?

I have the following named pipe created in Windows Powershell.
# .NET 3.5 is required to use the System.IO.Pipes namespace
[reflection.Assembly]::LoadWithPartialName("system.core") | Out-Null
$pipeName = "pipename"
$pipeDir = [System.IO.Pipes.PipeDirection]::InOut
$pipe = New-Object system.IO.Pipes.NamedPipeServerStream( $pipeName, $pipeDir )
Now, what i need is some Python code snippet to read from the above named pipe created. Can Python do that ?
Thanks in advance !
Courtesy :http://jonathonreinhart.blogspot.com/2012/12/named-pipes-between-c-and-python.html
Here's the C# Code
using System;
using System.IO;
using System.IO.Pipes;
using System.Text;
class PipeServer
{
static void Main()
{
var server = new NamedPipeServerStream("NPtest");
Console.WriteLine("Waiting for connection...");
server.WaitForConnection();
Console.WriteLine("Connected.");
var br = new BinaryReader(server);
var bw = new BinaryWriter(server);
while (true)
{
try
{
var len = (int)br.ReadUInt32(); // Read string length
var str = new string(br.ReadChars(len)); // Read string
Console.WriteLine("Read: \"{0}\"", str);
//str = new string(str.Reverse().ToArray()); // Aravind's edit: since Reverse() is not working, might require some import. Felt it as irrelevant
var buf = Encoding.ASCII.GetBytes(str); // Get ASCII byte array
bw.Write((uint)buf.Length); // Write string length
bw.Write(buf); // Write string
Console.WriteLine("Wrote: \"{0}\"", str);
}
catch (EndOfStreamException)
{
break; // When client disconnects
}
}
}
}
And here's the Python code:
import time
import struct
f = open(r'\\.\pipe\NPtest', 'r+b', 0)
i = 1
while True:
s = 'Message[{0}]'.format(i)
i += 1
f.write(struct.pack('I', len(s)) + s) # Write str length and str
f.seek(0) # EDIT: This is also necessary
print 'Wrote:', s
n = struct.unpack('I', f.read(4))[0] # Read str length
s = f.read(n) # Read str
f.seek(0) # Important!!!
print 'Read:', s
time.sleep(2)
Convert the C# code into a .ps1 file.

Categories

Resources