Develop Reference

Is there any feasible solution to read WOT battle results .dat files?

I am new here to try to solve one of my interesting questions in World of Tanks. I heard that every battle data is reserved in the client's disk in the Wargaming.net folder because I want to make a batch of data analysis for our clan's battle performances.
image
It is said that these .dat files are a kind of json files, so I tried to use a couple of lines of Python code to read but failed.
import json
f = open('ex.dat', 'r', encoding='unicode_escape')
content = f.read()
a = json.loads(content)
print(type(a))
print(a)
f.close()
The code is very simple and obviously fails to make it. Well, could anyone tell me the truth about that?
Added on Feb. 9th, 2022
After I tried another set of codes via Jupyter Notebook, it seems like something can be shown from the .dat files
import struct
import numpy as np
import matplotlib.pyplot as plt
import io
with open('C:/Users/xukun/Desktop/br/ex.dat', 'rb') as f:
fbuff = io.BufferedReader(f)
N = len(fbuff.read())
print('byte length: ', N)
with open('C:/Users/xukun/Desktop/br/ex.dat', 'rb') as f:
data =struct.unpack('b'*N, f.read(1*N))
The result is a set of tuple but I have no idea how to deal with it now.

Here's how you can parse some parts of it.
import pickle
import zlib
file = '4402905758116487.dat'
cache_file = open(file, 'rb') # This can be improved to not keep the file opened.
# Converting pickle items from python2 to python3 you need to use the "bytes" encoding or "latin1".
legacyBattleResultVersion, brAllDataRaw = pickle.load(cache_file, encoding='bytes', errors='ignore')
arenaUniqueID, brAccount, brVehicleRaw, brOtherDataRaw = brAllDataRaw
# The data stored inside the pickled file will be a compressed pickle again.
vehicle_data = pickle.loads(zlib.decompress(brVehicleRaw), encoding='latin1')
account_data = pickle.loads(zlib.decompress(brAccount), encoding='latin1')
brCommon, brPlayersInfo, brPlayersVehicle, brPlayersResult = pickle.loads(zlib.decompress(brOtherDataRaw), encoding='latin1')
# Lastly you can print all of these and see a lot of data inside.
The response contains a mixture of more binary files as well as some data captured from the replays.
This is not a complete solution but it's a decent start to parsing these files.

First you can look at the replay file itself in a text editor. But it won't show the code at the beginning of the file that has to be cleaned out. Then there is a ton of info that you have to read in and figure out but it is the stats for each player in the game. THEN it comes to the part that has to do with the actual replay. You don't need that stuff.
You can grab the player IDs and tank IDs from WoT developer area API if you want.

After loading the pickle files like gabzo mentioned, you will see that it is simply a list of values and without knowing what the value is referring to, its hard to make sense of it. The identifiers for the values can be extracted from your game installation:
import zipfile
WOT_PKG_PATH = "Your/Game/Path/res/packages/scripts.pkg"
BATTLE_RESULTS_PATH = "scripts/common/battle_results/"
archive = zipfile.ZipFile(WOT_PKG_PATH, 'r')
for file in archive.namelist():
if file.startswith(BATTLE_RESULTS_PATH):
archive.extract(file)
You can then decompile the python files(uncompyle6) and then go through the code to see the identifiers for the values.
One thing to note is that the list of values for the main pickle objects (like brAccount from gabzo's code) always has a checksum as the first value. You can use this to check whether you have the right order and the correct identifiers for the values. The way these checksums are generated can be seen in the decompiled python files.
I have been tackling this problem for some time (albeit in Rust): https://github.com/dacite/wot-battle-results-parser/tree/main/datfile_parser.

Tempfile path returning int python3

I would like to know if there is a way to find the path of a temporary file without it returning an int (15 in my case). I would like to return a string (or an object in which I can then turn into one, please detail how) path is the path to the temp file (and the name of it). Example: /Users/FooBar/Python3.6/Modules/TempFile/Opus/THE_TEMP_FILE or something like that. I have already written a small .wav file to it and would like to get the path to get the playing time/duration of it using os .stat(). I want to use a temporary file because I am lazy and I do not want to do a lot of 'special' code for the four different operating systems I am trying to this program on. Here is my code:
import pygame, time, os, tempfile
import urllib.request as ur
pygame.init() # Initialize the pygame
DISPLAYSURF = pygame.display.set_mode((1, 1)) # Sets display. Needed to play sound
sound = input('What sound should play: ') # Asks which sound
url = 'http://207.224.195.43:8000/' + sound # Gets server url
response = ur.urlopen(url) # Open url
data = response.read() # Create byte like object containing .wav code
f = tempfile.TemporaryFile() # Create TempFile
f.write(data) # Write .wav data gotten from server
f.seek(0) # Prepare to read it
soundObj = pygame.mixer.Sound(f.read()) # Load sound to be played
f.seek(0) # Prepare to read it
statbuf = os.stat(f.name) # Gets stats from TempFile. Returns int, I want to fix that
mbytes = statbuf.st_size / 1024 # Gets 'not real' sound duration
soundObj.play() # Plays sounds
time.sleep(mbytes / 200) # Gets 'real' sound duration and waits
soundObj.stop() # Stops sounds once done
Let me know (comment) if you have any suggestions. I have looked at a few sites, one of which was on stack-overflow, one that you could mistake to be a duplicate of this question. It was about Django which as I understand is entirely different then python. Thanks for possibly answering this question. Remember, I am not looking for a confirmation on this is an issue, I already know that. Please give me a possible answer to the question as soon as you can.
Thanks!
-User 9311010

The whole point of tempfile.TemporaryFile is that there is no name for the file, if at all possible:
… Under Unix, the directory entry for the file is either not created at all or is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having or not having a visible name in the file system.
If you want a temporary file with an accessible filename, use NamedTemporaryFile:
This function operates exactly as TemporaryFile() does, except that the file is guaranteed to have a visible name in the file system… That name can be retrieved from the name attribute of the returned file-like object.
However, I don't think you need a filename in the first place. Do you only want one so you have something to pass to stat? In Python 3.3+, you can pass a file object (like f) to stat instead of its name. In older versions, you can use fstat with the file's descriptor (fileno()).

Using Matlab Regex to insert "disclaimer" at begining of multiple codes within multiple subfolders

I have a folder with multiple subfolders that all contain several files. I am looking to write a matlab code that will insert a commented out "disclaimer" on the top of every relevant code [c, python (.py not .pyc), .urdf, .xml (.launch, .xacro, .config)]
My current thought process is to first list out every subfolder within the main folder. Then search within each subfolder for the relevant codes. If a relevant code is found, the disclaimer is commented in the top of the code... (each language has a different disclaimer)
I am having a hard time piecing this all together.. any help?
data_dir = 'C:thedirectorytomainfolder':
topLevelFolder = data_dir;
if topLevelFolder == 0
return;
end
% Get list of all subfolders.
allSubFolders = genpath(topLevelFolder);
remain = allSubFolders;
listOfFolderNames = {};
while true
[singleSubFolder, remain] = strtok(remain, ';');
if isempty(singleSubFolder)
break;
end
listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames)
%% Process all (wanted) files in those folders
for k = 1 : numberOfFolders
% Get this folder and print it out.
thisFolder = listOfFolderNames{k};
fprintf('Processing folder %s\n', thisFolder);
% Get .xml files.
filePattern = sprintf('%s/*.xml', thisFolder);
baseFileNames = dir(filePattern);
filePattern = sprintf('%s/*.c', thisFolder);
baseFileNames = [baseFileNames; dir(filePattern)];
numberOfImageFiles = length(baseFileNames)
I'm having a hard time reading each relevant file and inserting the correct comment code at the beginning of the file... any help?

Most of matlab's methods for reading text files assume you are trying to load in primarily numeric data but one of them might still work for you.
Sometimes it's easier to fopen the file and then read lines with fgetl of fread. Because you're doing low-level IO you have to test for the end of file too with while ~feof or somesuch. You could store each line in a cell array, prepend it with a cell array of your disclaimer and then write back out with fwrite, converting the cell back to a string with char.
It'll be pretty cumbersome. Does it have to be matlab? If you have the option it might be quicker to do it in a different language - it would be less than twenty lines in shell, and ruby/python/perl are all more geared up for text processing, which isn't matlab's strongest point.

python ringtone maker : how to split files

I'm writing an application for making ringtones for the iPhone. It's just for fun.
Here's what I've done till now. (Mind you, i'm a beginner in Python !)
So I open my mp3 file in binary mode. read the whole file. Convert it into a list. Use list slicing to split the file. Save this new split into a new mp3 file. This works fine. I however want to the ringtones to have a max of 30 seconds play time and I want the user to choose what portion of the file he wants as the ringtone. Can anyone guide me in the right direction ? Thanks
Here's my code till now :
f = open("Bruno Mars - Locked Out Of Heaven [OFFICIAL VIDEO].mp3", 'rb').read()
mp3 = list(f)
fo = open("newFile.mp3", "wb")
print(mp3[0:1300000])
fo.write(bytes(mp3[0:1300000]))
Here's what I got after some coding:
import os
f = open("Bruno Mars - Locked Out Of Heaven [OFFICIAL VIDEO].mp3", 'rb').read()
fileSize = os.path.getsize("Bruno Mars - Locked Out Of Heaven [OFFICIAL VIDEO].mp3")
print("Size of the whole file",fileSize)
mp3 = list(f)
bitRate = int(input("Enter the bit rate of your file"))
size_mbps = bitRate*(15/2048)
print("MB per minute :",size_mbps)
second_size = int((size_mbps/60)*(10**6))
print("Size of each second :",second_size)
start_length = int(input("Enter the start time (in seconds)"))
end_length = int(input("Enter the end time (in seconds)"))
start_size = int(second_size*start_length)
end_size = int(second_size*end_length)
fo = open("newFile.mp3", "wb")
fo.write(bytes(mp3[start_size:end_size]))
It works fine but I need to tweak it a little more. Any input on this code ?

If you want to use a higher-level language like c++ to implement a custom MP3 decoder, take a look a this question, time length of an mp3 file.
Otherwise, there are other libraries that do just what you need:
PyMad -
import mad
mf = mad.MadFile("foo.mp3")
track_length_in_milliseconds = mf.total_time()
audioread -
audio = audioread.audio_open('/path/to/mp3')
print f.channels, f.samplerate, f.duration
Mutagen -
from mutagen.mp3 import MP3
audio = MP3("example.mp3")
print audio.info.length
There are many more, but for now you can start with these.

You have to be a bit careful since mp3 frames do not always stand on their own, which is one of the reasons high end music tools will first convert to WAV or something like it, but as you remarked, it does work up till a certain level.
You could probably benefit from looking at the code in the following project (note that it is GPL licensed though):
http://sourceforge.net/projects/pymp3cut/
the interesting bits:
http://pymp3cut.cvs.sourceforge.net/viewvc/pymp3cut/pymp3cut/pmpcmp3.py?revision=1.2&view=markup
Once you extracted the actual music part, the rough idea is that you base yourself on the bitrate to get the relation between the framesize, seconds, filesize and byte content, which then tells you where to split the bytes to get the starting time and duration that you want.

Linux and python: Combining multiple wave files to one wave file

I am looking for a way that I can combine multiple wave files into one wave file using python and run it on linux. I don't want to use any add on other than the default shell command line and default python modules.
For example, if I have a.wav and b.wav. I want to create a c.wav which start with the content from a.wav then b.wav.
I've found wave module, that I can open a wave file and write into a new file. Since i'm really new in this audio world. I still can't figure out how to do it. Below is my code
import struct, wave
waveFileA = wave.open('./a.wav', 'r')
waveFileB = wave.open('./b.wav', 'r')
waveFileC = wave.open('./c.wav', 'w')
lengthA = waveFileA.getnframes()
for i in range(0,lengthA):
waveFileC.writeframes(waveFileA.readframes(1))
lengthB = waveFileB.getnframes()
for i in range(0,lengthB):
waveFileC.writeframes(waveFileB.readframes(1))
waveFileA.close()
waveFileB.close()
waveFileC.close()
When i run this code, I got this error:
wave.Error: # channels not specified
Please can any one help me?

You need to set the number of channels, sample width, and frame rate:
waveFileC.setnchannels(waveFileA.getnchannels())
waveFileC.setsampwidth(waveFileA.getsampwidth())
waveFileC.setframerate(waveFileA.getframerate())
If you want to handle a.wav and b.wav having different settings, you'll want to use something like pysox to convert them to the same settings, or for nchannels and sampwidth you may be able to tough through it yourself.

Looks like you need to call n=waveFileA.getnchannels() to find out how many channels the first input file uses, likewise for waveFileB, then you'll need to use waveFileC.setnchannels(n) to tell it how many channels to put in the outgoing file. I don't know how it will handle input files with different numbers of channels...

Here is the answer I am looking for
How to join two wav files using python?
(look for a thread by Tom 10)
It's in another thread. some one already solved this problem.