I have a snippet of a code written in python which I have to rewrite to C# with the help of Newtonsoft JSON
def getconfig():
with open("config.json") as f:
return json.loads(f.read())
and I have to be able to access the elements of JSON in the code just as it's shown in this snippet
def getcaptcha(proxy):
while(True):
ret = s.get("{}{}&pageurl={}"
.format(getconfig()['host'], getconfig()['key'], getconfig()['googlekey'], getconfig()['pageurl'])).text
So far I've come up with this idea:
public bool GetConfig()
{
JObject config = JObject.Parse(File.ReadAllText("path/config.json"));
using (StreamReader file = File.OpenText("path/config.json"))
using (JsonTextReader reader = new JsonTextReader(file))
{
JObject getConfig = (JObject) JToken.ReadFrom(reader);
}
return true;
}
but it doesn't seem to be working since I can't access it in further code as
GetConfig()['host']
for example. I can't even Console.WriteLine my JSON
First, define your GetConfig function as:
public JObject GetConfig()
{
return JObject.Parse(File.ReadAllText("path/config.json"));
}
then, call that function and access elements defined in the json file like:
JObject config = GetConfig();
string host = config.SelectToken("host").Value<string>();
string key = config.SelectToken("key").Value<string>();
Related
I try to extract two kind of value from a txt file and write them to two separate txt files. I know that my functions work properly and I cannot figure out any mistake in my code. I realised that both two languages do not read the text file as it is. What I mean by that is for example normally the txt file has 10367 lines in it but when I count the lines in the code, there are 20735 lines in python. I cannot understand why this happens. I do not have an in-depth knowledge about how programming languages read the files. Please give me some information about the possible causes of this situation.
thanks in advance...
This is the pyhton code:
def main():
serverSpeedsList=list()
totalSpeedsList=list()
ssString=str()
stString=str()
with open("C:\\Users\\yusuf\\OneDrive\\Masaüstü\\SpeedTests\\Logs\\log100.txt",'r') as inFile:
for line in inFile:
i+=1
ss=speedOfServer(line)
st=speedOfTotal(line)
if ss!="":
ssString+=ss+"\n"
serverSpeedsList.append(ss)
if st!="":
stString+=st+"\n"
totalSpeedsList.append(st)
with open("C:\\Users\\yusuf\\OneDrive\\Masaüstü\\SpeedTests\\Results\\server100.txt",'w') as outFile:
outFile.write(ssString)
with open("C:\\Users\\yusuf\\OneDrive\\Masaüstü\\SpeedTests\\Results\\total100.txt",'w') as outFile:
outFile.write(ssString)
def speedOfServer(text):
startStr="time\":\""
endStr=" ms"
result=str()
startIx=text.find(startStr)
endIx=text.find(endStr)
if startIx>=0 and endIx>=0:
result=text[startIx+len(startStr) : endIx]
return result
def speedOfTotal(text):
startStr="showProfile.php ("
endStr="ms"
result=str()
startIx=text.find(startStr)
endIx=text.find(endStr)
if startIx>=0 and endIx>=0:
result=text[startIx+len(startStr) : endIx]
return result
main()
and this is the java code to do the same
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
public class Main {
private static String serverSpeedFileName="C:\\Users\\yusuf\\OneDrive\\Masaüstü\\JavaSpeedAnalyser\\Results\\Server\\serverSpeed100.txt";
private static String responseSpeedFileName="C:\\Users\\yusuf\\OneDrive\\Masaüstü\\JavaSpeedAnalyser\\Results\\Total\\responseSpeed100.txt";
private static String logFilePath="C:\\Users\\yusuf\\OneDrive\\Masaüstü\\JavaSpeedAnalyser\\Logs\\log100.txt";
public static void main(String[] args){
StringBuilder serverSpeeds=new StringBuilder();
StringBuilder responseSpeeds=new StringBuilder();
try{
File file = new File(logFilePath);
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null){
String serverSpeed=speedOfServer(line);
if(!serverSpeed.isEmpty()){
System.out.println(serverSpeed);
serverSpeeds.append(serverSpeed+"\n");
}
String responseSpeed=speedOfResponses(line);
if(!responseSpeed.isEmpty()){
responseSpeeds.append(responseSpeed+"\n");
}
}
br.close();
}catch(IOException e){
e.printStackTrace();
}
writeToFile(serverSpeedFileName, serverSpeeds.toString());
writeToFile(responseSpeedFileName, responseSpeeds.toString());
}
private static String speedOfServer(String text){
String start="time\":\"";
String end=" ms";
String result="";
int startIndex=text.indexOf(start);
int endIndex=text.indexOf(end);
if(startIndex>=0 && endIndex>=0 ){
result=text.substring(startIndex+start.length(),endIndex);
}
return result;
}
private static String speedOfResponses(String text){
String start="%5bF%5dshowProfile.php (";
String end="ms)";
String result="";
int startIndex=text.indexOf(start);
int endIndex=text.indexOf(end);
if(startIndex>=0 && endIndex>=0){
result=text.substring(startIndex+start.length(),endIndex);
}
return result;
}
}
I try to analyse a logcat file from an android phone, this is why I try to do that but I cannot manage it. Please help me
I have a python script that reads from a logfile and outputs certain data from it. The way it reads from it is
try:
with open(os.path.expandvars('Path/To/My/Log.txt', 'r') as f:
logContent = [line.rstrip() for line in f]
except Exception as e:
print(e)
Now I wanted to recreate that python script in C#. The main problem is, that the log file makes about 30.000 Lines in 30 minutes. While the program that handles that log isn't being executed, I can easily open the file and read from it, because it's not being used by that program. But when that program runs, I need to read from the file with a filestream, and so the reading of 30.000 lines takes ages:
private string GetLog(string path)
{
string log = "";
FileStream reader = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
StreamReader logFileReader = new StreamReader(reader);
while (!logFileReader.EndOfStream)
{
log += logFileReader.ReadLine();
// Your code here
}
// Clean up
logFileReader.Close();
reader.Close();
return log;
}
Is there a way to make my code read from the file in max 5 seconds?
I got it. When I use stream.ReadToEnd() it reads everything in about 2 seconds
As you have mentioned file is big, so better to use StringBuilder over string, you can use using also so no need to call close() explicitly.
StringBuilder sb = new StringBuilder();
string path = "some path";
using (FileStream logFileStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (StreamReader logFileReader = new StreamReader(logFileStream))
{
while (!logFileReader.EndOfStream)
{
sb.Append(logFileReader.ReadLine());
}
}
}
string log = sb.ToString();
I would like to use automation to create the hocon configuration with python 3 scripting. I read that lightbend (https://github.com/lightbend/config) recommends pyhocon (https://github.com/chimpler/pyhocon).
I am having problems figuring out how to create an Hocon object and write the data to a file as hocon. It is important to me that the syntax for the substitution are in the result.
For example I expect the output of the file myconfig.conf to look something like this:
{
Environment: "dev"
JobName: ${Environment}"-hello-bob"
}
So, I assumed that there was a way to do something like this:
config2 = ConfigFactory.parse_string("{}")
config2.put("Environment", "dev")
#Some type of object for references or special syntax for ${Environment}
config2.put("JobName", "${Environment}")
Then after creating the stuffed object there should be a simple way to write out to a file or files:
filename = "myconfig.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
fd.write(config2.to_hocon_str)
Has anyone figured a way to do this? It seems odd that the library can only be used for reading data only.
Probably this example will answer your question
from pyhocon.converter import HOCONConverter
import pyhocon
string = '{"Environment": "Dev","Test": ${Environment}}'
factory = pyhocon.ConfigFactory.parse_string(string, resolve=True)
factory.put('somekey','somevalue')
print(HOCONConverter().to_hocon(factory))
returns
Environment = "Dev"
Test = "Dev"
somekey = "somevalue"
So, I decided to look at documentation for JVM (Java/Scala) library (https://github.com/lightbend/config). After reading the documentation, there was a clear section on hocon examples (https://github.com/lightbend/config#examples-of-hocon). In this documentation, they categorized 7 valid hocon styles. I call these styles because if I was to automate the generation of these files, I would be picking one way to write out and sticking with it.
All of these are valid HOCON.
1.Start with valid JSON:
{
"foo" : {
"bar" : 10,
"baz" : 12
}
}
2.Drop root braces:
"foo" : {
"bar" : 10,
"baz" : 12
}
3.Drop quotes:
foo : {
bar : 10,
baz : 12
}
4.Use = and omit it before {:
foo {
bar = 10,
baz = 12
}
5.Remove commas:
foo {
bar = 10
baz = 12
}
6.Use dotted notation for unquoted keys:
foo.bar=10
foo.baz=12
7.Put the dotted-notation fields on a single line:
foo.bar=10, foo.baz=12
Because I will be using the pyhocon library, I needed to look for write solutions within the library. I found some help from chimpler's git (https://github.com/chimpler/pyhocon). What I found was that they have two hocon styles which can be simply written out. One is json and the other is something that wasn't on the list which was describe above by lightbend.
Style 1: pure JSON, witch can be written out in two ways:
HOCONConverter.to_json
#Using HOCONConverter.to_json
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./json_coverted.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
fd.write(HOCONConverter.to_json(confTree))
HOCONConverter.to_json Result
{
"Environment": "Dev",
"Test": "${Environment}"
}
OR Using json.dump
#Using json.dump
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./json_dumped.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
fd.write(json.dumps(confTree,indent=4))
Using json.dump Result
{
"Environment": "Dev",
"Test": "${Environment}"
}
Pyhocon's other Style, not listed by lightbend
# HOCONConverter.to_hocon
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./hocon_coverted.txt"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
fd.write(HOCONConverter.to_hocon(confTree))
Pyhocon's other Style, not listed by lightbend Result
Environment = "Dev"
Test = "${Environment}"
So, to answer my own question the only dependable way to generate a hocon conf file dynamically using pyhocon in Python 3 is by using one of the json methods (converter or dumps). But this still leaves an open question. The question being, will reading a json to a pyhocon ConfTree object be able dereference the substitutions when they are in the json?
For example if I read the file
{
"Environment": "Dev",
"Test": "${Environment}"
}
Will the ConfTree object get "Dev" as the value for Test?
No, it will not. Here is my test
filename = "json_coverted.conf"
print("Reading file{}".format(filename))
conf = ConfigFactory.parse_file(filename)
key="Test"
value=conf.get(key)
print("Key:{} Value:{}".format(key,value))
Test Result Out to screen
Reading filejson_coverted.conf
Key:Test Value:${Environment}
So, then how does one use pyhocon with substitutions?
It just can't hence, I will not use either library for writing out confs. It has to be a manual process if I want to use substitutions. So, I am only using this library for reading confs.
I need to open moz-extension://internal-uuid page after my Selenium script is started to have an access to the extension's storage API, to set some prefs there, that this extension will read later and use to do some actions. But when I use selenium.webdriver.Firefox.add_addon(...) it returns the Extension ID that differs and can't be used to open the moz-extension:// page. Is there any way to get this Internal UUID from my code (not manually by inspecting about:debugging#addons). Or may be some way to pass the data I need from Selenium to Web Extension?
This code is working for me in Linux and Mac:
public static void main(String[] args) throws IOException {
FirefoxOptions options = new FirefoxOptions();
FirefoxDriver driver = new FirefoxDriver(options);
String userPrefsFileContent = readFile(driver.getCapabilities().getCapability("moz:profile") + "/prefs.js");
String extensionUuid = getExtensionUuid(userPrefsFileContent);
driver.quit();
}
private static String getExtensionUuid(String userPrefsFileContent) {
String uuid = null;
String[] usersPrefsList = userPrefsFileContent.split(";");
for (String currentPref : usersPrefsList) {
if (currentPref.contains("extensions.webextensions.uuids")) {
uuid = currentPref.split(":")[1].replaceAll("\"", "").replace("}", "")
.replace(")", "").replace("\\", "");
}
}
if(uuid.contains(",")) {
uuid = uuid.split(",")[0];
}
return uuid;
}
private static String readFile(String pathname) throws IOException {
File file = new File(pathname);
StringBuilder fileContents = new StringBuilder((int) file.length());
String lineSeparator = System.getProperty("line.separator");
try (Scanner scanner = new Scanner(file)) {
while (scanner.hasNextLine()) {
fileContents.append(scanner.nextLine()).append(lineSeparator);
}
}
return fileContents.toString();
}
Don't know how, but I cannot get this approach to work in python, so instead found out another smart approach, instead of trying to get preference that is not provided by Firefox, you can set UUID to something with set-preference API. then just use it.
here is part of my code
if self.webdriver != None:
return self.webdriver
extensionPath = curPath+'/../../packages/firefox.xpi';
cap = DesiredCapabilities().FIREFOX
cap["marionette"]=True;
profile = webdriver.firefox.firefox_profile.FirefoxProfile();
profile.set_preference('extensions.webextensions.uuids',json.dumps({'support#ipburger.com':firefoxUUID}))
self.webdriver = webdriver.Firefox(firefox_binary=firefoxBinary,capabilities=cap,executable_path=curPath+'/../../drivers/geckodriver',firefox_profile=profile);
self.webdriver.install_addon(extensionPath,temporary=True);
return self.webdriver;
firefoxUUID is a string version of random UUID you generate
and you have to replace support#ipburger.com with your addon ID, which you can set inside manifest.json file
the code below is working for me in python on windows :
driver = webdriver.Firefox(service=firefox_service, options=firefox_options)
driver.install_addon(os.path.join(application_path, 'extension_firefox.xpi'), temporary=True)
time.sleep(1)
profile_path = driver.capabilities['moz:profile']
with open('{}/prefs.js'.format(profile_path), 'r') as file_prefs:
lines = file_prefs.readlines()
for line in lines:
if 'extensions.webextensions.uuids' in line:
sublines = line.split(',')
for subline in sublines:
if id_extension_firefox in subline:
internal_uuid = subline.split(':')[1][2:38]
I have 5gb of data serialized with apache thrift and a .thrift file with the formatting of the data. I have tried using thriftpy and the official thrift package but I can't wrap my head around how to open the files.
The data is the expanded dataset from http://www.iesl.cs.umass.edu/data/wiki-links
A description of the data format can be found here https://code.google.com/p/wiki-link/wiki/ExpandedDataset
The Scala setup is to be found in the ThriftSerializerFactory.scala file. Since the naming of most things is consistent throughout the Thrift libraries, you more or less model your python code after the Scala example:
package edu.umass.cs.iesl.wikilink.expanded.process
import org.apache.thrift.protocol.TBinaryProtocol
import org.apache.thrift.transport.TIOStreamTransport
import java.io.File
import java.io.BufferedOutputStream
import java.io.FileOutputStream
import java.io.BufferedInputStream
import java.io.FileInputStream
import java.util.zip.{GZIPOutputStream, GZIPInputStream}
object ThriftSerializerFactory {
def getWriter(f: File) = {
val stream = new BufferedOutputStream(new GZIPOutputStream(new FileOutputStream(f)), 2048)
val protocol= new TBinaryProtocol(new TIOStreamTransport(stream))
(stream, protocol)
}
def getReader(f: File) = {
val stream = new BufferedInputStream(new GZIPInputStream(new FileInputStream(f)), 2048)
val protocol = new TBinaryProtocol(new TIOStreamTransport(stream))
(stream, protocol)
}
}
You basically set up a stream transport and the binary protocol. If you leave the data compressed, you will have to add the gzip piece to the puzzle, but once the data are decompressed this should not be needed anymore.
The code in WikiLinkItemIterator.scala shows how to read the data files using the factory class above.
class PerFileWebpageIterator(f: File) extends Iterator[WikiLinkItem] {
var done = false
val (stream, proto) = ThriftSerializerFactory.getReader(f)
private var _next: Option[WikiLinkItem] = getNext()
private def getNext(): Option[WikiLinkItem] = try {
Some(WikiLinkItem.decode(proto))
} catch {case _: TTransportException => {done = true; stream.close(); None}}
def hasNext(): Boolean = !done && (_next != None || {_next = getNext(); _next != None})
def next(): WikiLinkItem = if (hasNext()) _next match {
case Some(wli) => {_next = None; wli}
case None => {throw new Exception("Next on empty iterator.")}
} else throw new Exception("Next on empty iterator.")
}
Steps to implement:
implement Thrift protocol stack factory like above (recommendable pattern, BTW)
instantiate the root element of each record, in our case a WikiLinkItem
call instance.read(proto) to read one record of data