This question is an evolution of this: Issue returning header byte from Python to Unity
Now the issue about header byte is solved, but I have a problem during the loop.
The data incoming good but after "n" times the loop break because receive a json string with an entire structure of data with open/close bracket PLUS the next structure of data with open bracket, some data and in the "image:" some data and many of the "\0" and unless the close bracket.. And the json parser obviously breaks. Why and what can I do? Thanks
Below the code revised:
IEnumerator Client()
{
while (!IsConnected(client))
{
try
{
client = new TcpClient(host, port);
s = client.GetStream();
byte[] byteBuffer = Encoding.UTF8.GetBytes("Connected to client");
s.Write(byteBuffer, 0, byteBuffer.Length);
while (true)
{
if (s.DataAvailable)
{
while (s.DataAvailable)
{
var header = new byte[4];
s.Read(header, 0, header.Length);
var fileSize = BitConverter.ToUInt32(header,0);
StringBuilder myCompleteMessage = new StringBuilder();
MemoryStream ms = new MemoryStream();
int increment = 0;
while (ms.Length < fileSize)
{
byte[] dataReceived = new byte[fileSize];
increment = s.Read(dataReceived, 0, dataReceived.Length);
ms.Write(dataReceived.Take(increment).ToArray(), 0, increment);
}
myCompleteMessage.AppendFormat("{0}", Encoding.UTF8.GetString(ms.GetBuffer()));
JSONNode data = JSONNode.Parse(myCompleteMessage.ToString());
....
// Do work with myCompleteMessage
yield return null;
}
}
}
}
}
}
I don't have your Unity environment here, so I can't have tested this.
Something like a loop (likely run in a coroutine or something that doesn't block Unity's default handling) like this should be enough to handle a stream of messages of the shape we established before (a 4-byte header signifying the message length, followed by that many bytes of JSON).
while (!IsConnected(client))
{
try
{
client = new TcpClient(host, port);
s = client.GetStream();
while (true)
{
var data = ReadPacket(s);
// Do work with data
}
}
}
ReadPacket should be something like
JSONNode ReadPacket(Stream s)
{
var buffer = new byte[131072];
// Read 4 bytes (we will assume we can always read that much)
var n = s.Read(buffer, 0, 4);
if(n < 4) throw new Exception("short read");
var fileSize = BitConverter.ToUInt32(buffer, 0);
Debug.Print("Reading a blob of length", fileSize);
// Memory stream to accumulate the reads into.
var ms = new MemoryStream();
while (ms.Length < fileSize)
{
// Figure out how much we can read -- if we're near the end,
// don't overread.
var maxRead = Math.Min(buffer.Length, fileSize - ms.Length);
var increment = s.Read(buffer, 0, maxRead);
// ... and write them into the stream.
ms.Write(buffer, 0, increment);
Debug.Print("Read", ms.Length, "of", fileSize);
}
// Decode the bytes to UTF-8 and parse.
return JSONNode.Parse(Encoding.UTF8.GetString(ms.GetBuffer()));
}
Related
I am writing a program that requires the starting of a python script before the rest of the java code runs. However, I cannot find a solution to my issues. I would appreciate if someone could suggest a solution to the problem I am facing.
Code (I need help on the part under the comment "start python"):
import java.io.IOException;
//makes it easier for user to
//select game/start python
public class gameselect {
public static void main(String args[]) throws IOException {
//start python
try {
String cmd = "python ngramcount.py";
Process process = Runtime.getRuntime().exec(cmd);
process.getInputStream();
}
catch (IOException e) {
e.printStackTrace();
}
//select game
try {
Scanner in = new Scanner (System.in);
game1 g = new game1();
game2 f = new game2();
int choice = 0;
System.out.println("Welcome to TranslateGame!");
System.out.println("Type 1 for game1 (words) or 2 for game2 (phrases)");
while (choice != 1 && choice != 2) {
choice = in.nextInt();
if (choice != 1 && choice != 2) {
System.out.println("No game associated with that number.");
}
}
if (choice == 1) {
g.game1();
}
else if (choice == 2) {
f.game2();
}
}
catch(IOException e) {
System.out.println("No.");
}
}
}
Here is some code that you might be able to get to work. I also commented it and provided some reference links to help you understand what the code is doing.
public static void main(String[] args) throws IOException {
// I'm using the absolute path for my example.
String fileName = "C:\\Users\\yourname\\Desktop\\testing.py";
// Creates a ProcessBuilder
// doc: https://docs.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html
ProcessBuilder pb = new ProcessBuilder("python", fileName);
pb.redirectErrorStream(true); // redirect error stream to a standard output stream
Process process = pb.start(); // Used to start the process
// Reads the output stream of the process.
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line; // this will be used to read the output line by line. Helpful in troubleshooting.
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}
This code coverts an Array Buffer to a String and vice versa
function ab2str(buf) {
return String.fromCharCode.apply(null, new Uint16Array(buf));
}
function str2ab(str) {
var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
var bufView = new Uint16Array(buf);
for (var i=0, strLen=str.length; i < strLen; i++) {
bufView[i] = str.charCodeAt(i);
}
return buf;
}
I think Python bytearray() can do the job.
u can refer this page for more info: https://www.programiz.com/python-programming/methods/built-in/bytearray
I'm using the below lines of code to extract the pixel array from a camera at every frame, saving it as jpg and then running a python process on the jpg. Although it works, it is incredibly slow. The bottleneck seems to be reading the pixels in unity. The python process itself only lasts 0.02 seconds.
Can anyone suggest relatively easy ways I can speed up this process?
I've seen this , but it's too high-level for me to understand how to adapt it to my use-case.
public override void Initialize()
{
renderTexture = new RenderTexture(84, 84, 24);
rawByteData = new byte[84 * 84 * bytesPerPixel];
texture2D = new Texture2D(84, 84, TextureFormat.RGB24, false);
rect = new Rect(0, 0, 84, 84);
cam.targetTexture = renderTexture;
}
private List<float> run_cmd()
{
// Setup a camera, texture and render texture
cam.targetTexture = renderTexture;
cam.Render();
// Read pixels to texture
RenderTexture.active = renderTexture;
texture2D.ReadPixels(rect, 0, 0);
rawByteData = ImageConversion.EncodeToJPG(texture2D);
// Assign random temporary filename to jpg
string fileName = "/media/home/tmp/" + Guid.NewGuid().ToString() + ".jpg";
File.WriteAllBytes(fileName, rawByteData); // Requires System.IO
// Start Python process
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = "/media/home/path/to/python/exe";
start.Arguments = string.Format(
"/media/home/path/to/pythonfile.py photo {0}", fileName);
start.UseShellExecute = false;
start.RedirectStandardOutput = true;
start.RedirectStandardError = true;
string stdout;
using(Process process = Process.Start(start))
{
using(StreamReader reader = process.StandardOutput)
{
stdout = reader.ReadToEnd();
}
}
string[] tokens = stdout.Split(',');
List<float> result = tokens.Select(x => float.Parse(x)).ToList();
System.IO.File.Delete(fileName);
return result;
}
In Unity 2018.1 was added a new system for async read data from GPU - https://docs.unity3d.com/ScriptReference/Rendering.AsyncGPUReadback.html, and in this not necessary use camera for getting array of textures, I wrote a simple example which good work and pretty fast:
using System.Collections;
using System.IO;
using System.Threading.Tasks;
using UnityEngine;
using UnityEngine.Experimental.Rendering;
using UnityEngine.Rendering;
public class CameraTextureTest : MonoBehaviour
{
private GraphicsFormat format;
private int frameIndex = 0;
IEnumerator Start()
{
yield return new WaitForSeconds(1);
while (true)
{
yield return new WaitForSeconds(0.032f);
yield return new WaitForEndOfFrame();
var rt = RenderTexture.GetTemporary(Screen.width, Screen.height, 32);
format = rt.graphicsFormat;
ScreenCapture.CaptureScreenshotIntoRenderTexture(rt);
AsyncGPUReadback.Request(rt, 0, TextureFormat.RGBA32, OnCompleteReadback);
RenderTexture.ReleaseTemporary(rt);
}
}
void OnCompleteReadback(AsyncGPUReadbackRequest request)
{
if (request.hasError)
{
Debug.Log("GPU readback error detected.");
return;
}
byte[] array = request.GetData<byte>().ToArray();
Task.Run(() =>
{
File.WriteAllBytes($"D:/Screenshots/Screenshot{frameIndex}.png",ImageConversion.EncodeArrayToPNG(array, format, (uint) Screen.width, (uint) Screen.height));
});
frameIndex++;
}
}
You can adapt this example to your task.
Now when I understand how the code works, I would like to translate it to C++.
The original Python code:
def recv_all_until(s, crlf):
data = ""
while data[-len(crlf):] != crlf:
data += s.recv(1)
return data
Here's what I tried:
std::string recv_all_until(int socket, std::string crlf)
{
std::string data = "";
char buffer[1];
memset(buffer, 0, 1);
while(data.substr(data.length()-2, data.length()) != crlf)
{
if ((recv(socket, buffer, 1, 0)) == 0)
{
if (errno != 0)
{
close(socket);
perror("recv");
exit(1);
}
}
data = data + std::string(buffer);
memset(buffer, 0, 1);
}
return data;
}
But it shows:
terminate called after throwing an instance of 'std::out_of_range'
what(): basic_string::substr
I understand that the problem is inside the while loop since at first the data string is empty. So how to improve this to make it work the same as it works in Python? Thank you.
You have the problem in the first iteration of your while loop:
Since the data is an empty string, data.length() is equal to 0, and thus you're calling data.substr(-2, 0).
To fix this, you need to add a check for the line length to the while statement.
Also, there's a method of finding such mistakes faster than writing a stackoverflow question about it. Consider reading this article.
If we first change your Python code a bit:
def recv_all_until(s, crlf):
data = ""
while not data.endswith(crlf):
data += s.recv(1)
return data
What we need to do in C++ becomes much clearer:
bool ends_with(const std::string& str, const std::string& suffix)
{
return str.size() >= suffix.size() &&
std::equal(suffix.rbegin(), suffix.rend(), str.rbegin());
}
std::string recv_all_until(int socket, const std::string& crlf)
{
std::string data = "";
char buffer[1];
memset(buffer, 0, 1);
while (!ends_with(data, crlf))
{
if ((recv(socket, buffer, 1, 0)) == 0)
{
if (errno != 0)
{
close(socket);
perror("recv");
exit(1);
}
}
data = data + std::string(buffer);
memset(buffer, 0, 1);
}
return data;
}
I'd like to be able to detect progressive jpegs using python and convert them to non progressive.
(I'm writing a tool to manage images for android and progressive jpegs seem to break it.)
I apologise in advance for providing a php based answer, whereas the question was asked about python. Nevertheless, I think it adds value and can be useful.
Before attempting to convert a progressive image to non-progressive, it is good to have a detection method for progressive Jpeg.
Here is the php function that does it, could easily be rewritten in other languages (python would be a candidate), as it reads binary data and Jpeg markers (and therefore does not rely on language specific library)
public function checkProgressiveJPEG($filepath) {
$result = false;
// $this->log = 'started analysis...';
// http://en.wikipedia.org/wiki/Jpeg
// for more details on JPEG structure
// SOI [0xFF, 0xD8] = Start Of Image
// SOF0 [0xFF, 0xC0] = Start Of Frame (Baseline DCT)
// SOF2 [0xFF, 0xC2] = Start Of Frame (Progressive DCT)
// SOS [0xFF, 0xDA] = Start Of Scan
if(file_exists($filepath)) {
$fs = #fopen($filepath, "rb");
$bytecount = 0;
$byte_last = 0;
$buffer = 0;
$buffer_length = 4*1024;
$begins_with_SOI = false;
while($buffer = fread($fs, $buffer_length)) {
// always carry over previous ending byte
// just in case the buffer is read after a 0xFF marker
if($byte_last) {
$buffer = $byte_last.$buffer;
}
$byte_last = 0;
preg_match("/\.$/", $buffer, $matches);
if(count($matches)) {
$byte_last = $matches[0];
}
// check if it begins with SOI marker
if(!$begins_with_SOI) {
preg_match("/^\\xff\\xd8/", $buffer, $matches);
if(count($matches)) {
$begins_with_SOI = true;
} else {
// $this->log = 'does not start with SOI marker';
$result = false;
break;
}
}
// check if SOS or SOF2 is reached
preg_match("/\\xff(\\xda|\\xc2)/", $buffer, $matches);
if(count($matches)) {
if(bin2hex($matches[0]) == 'ffda') {
// $this->log = 'SOS is reached and SOF2 has not been detected, so image is not progressive.';
$result = false;
break;
} else if(bin2hex($matches[0]) == 'ffc2') {
// $this->log = 'SOF2 is reached, so image is progressive.';
$result = true;
break;
}
}
} // end while
fclose($fs);
} // end if
return $result;
}