I am new to python/GDAL and am running into perhaps a trivial issue. This may stem from the fact that I don't really understand how to use GDAL properly in python, or something careless, but even though I think I am following the help doc, I keep getting a syntax error when trying to use "gdalbuildvrt".
What I want to do is take several (amount varies for each set, call it N) geotagged 1-band binary rasters [all values are either 0 or 1] of different sizes (each raster in the set overlaps for the most part though), and "stack" them on top of each other so that they are aligned properly according to their coordinate information. I want this "stack" simply so I can sum the values and produce a 'total' tiff that has an extent to match the exclusive extent (meaning not just the overlap region) of all the original rasters. The resulting tiff would have values ranging from 0 to N, to represent the total number of "hits" the pixel in that location received over the course of the N rasters.
I was led to gdalbuildvrt [http://www.gdal.org/gdalbuildvrt.html] and after reading about it, it seemed that by using the keyword -separate, I would be able to achieve what I need. However, each time I try to run my program, I get a syntax error. The following shows two of the several different ways I tried calling gdalbuildvrt:
gdalbuildvrt -separate -input_file_list stack.vrt inputlist.txt
gdalbuildvrt -separate stack.vrt inclassfiles
Where inputlist.txt is a text file with a path to the tif on every line, just like the help doc specifies. And inclassfiles is a python list of the pathnames. Every single time, no matter which way I call it, I get a syntax error on the first word after the keywords (i.e. 'inputlist' in inputlist.txt, or 'stack' in stack.vrt).
Could someone please shed some light on what I might be doing wrong? Alternatively, does anyone know how else I could use python to get what I need?
Thanks so much.
gdalbuildvrt is a GDAL command line utility. From your example its a bit unclear how you actually run it, but when running from within Python you should execute it as a subprocess.
And in your first line you have the .vrt and the .txt in the wrong order. The textfile containing the files should follow directly after the -input_file_list.
From within Python you can call gdalbuildvrt like:
import os
os.system('gdalbuildvrt -separate -input_file_list inputlist.txt stack.vrt')
Note that the command is provided as a string. Using a Python list with the files can be done with something like:
os.system('gdalbuildvrt -separate stack.vrt %s') % ' '.join(data)
The ' '.join(data) part converts the list to a string with a space between the items.
Depending on how your GDAL is build, its sometimes possible to use wildcards as well:
os.system('gdalbuildvrt -separate stack.vrt *.tif')
Related
I am trying to read sav files using pyreadstat in python but for some rare scenarios I am getting error of UnicodeDecodeError since the string variable has special characters.
To handle this I think instead of loading the entire variable set I will load only variables which do not have this error.
Below is the pseudo-code that I have with me. This is not a very efficient code since I check for error in each item of list using try and except.
# Reads only the medata to get information about the variables
df, meta = pyreadstat.read_sav('Test.sav', metadataonly=True)
list = meta.column_names # All variables are stored in list
result = []
for var in list:
print(var)
try:
df, meta = pyreadstat.read_sav('Test.sav', usecols=[str(var)])
# If no error that means we can store this variable in result
result.append(var)
except:
pass
# This will finally load the sav for non error variables
df, meta = pyreadstat.read_sav('Test.sav', usecols=result)
For a sav file with 1000+ variables it takes a long amount of time to process this.
I was thinking if there is a way to use divide and conquer approach and do it faster. Below is my suggested approach but I am not very good in implementing recursion algorithm. Can someone please help me with pseudo code it would be very helpful.
Take the list and try to read sav file
In case of no error then output can be stored in result and then we read the sav file
In case of error then split the list into 2 parts and run these again ....
Step 3 needs to run again until we have a list where it does not give any error
Using the second approach 90% of my sav files will get loaded on the first pass itself hence I think recursion is a good method
You can try to reproduce the issue for sav file here
For this specific case I would suggest a different approach: you can give an argument "encoding" to pyreadstat.read_sav to manually set the encoding. If you don't know which one it is, what you can do is iterate over the list of encodings here: https://gist.github.com/hakre/4188459 to find out which one makes sense. For example:
# here codes is a list with all the encodings in the link mentioned before
for c in codes:
try:
df, meta = p.read_sav("Test.sav", encoding=c)
print(encoding)
print(df.head())
except:
pass
I did and there were a few that may potentially make sense, assuming that the string is in a non-latin alphabet. However the most promising one is not in the list: encoding="UTF8" (the list contains UTF-8, with dash and that fails). Using UTF8 (no dash) I get this:
నేను గతంలో వాడిన బ
which according to google translate means "I used to come b" in Telugu. Not sure if that fully makes sense, but it's a way.
The advantage of this approach is that if you find the right encoding, you will not be loosing data, and reading the data will be fast. The disadvantage is that you may not find the right encoding.
In case you would not find the right encoding, you anyway would be reading the problematic columns very fast, and you can discard them later in pandas by inspecting which character columns do not contain latin characters. This will be much faster than the algorithm you were suggesting.
rebar_set_command=[]
for i in rebar_instances:
rebar_set_command.append('m.rootAssembly.instances[\''+i+'\'].faces.getByBoundingBox(0,0,0,X,Y,Z')
a='+'.join(rebar_set_command)
m.rootAssembly.Set(faces=a, name='A')
However it cannot be done because I think the value a in faces=a contains quotation marks.
How can I call the string to this Abaqus command without the opening and closing quote? Thanks!
The problem is 'm.rootAssembly.instances[\''+i+'\'].faces.getByBoundingBox(0,0,0,X,Y,Z' gives you a string. It's not a Python expression. You may want to check out some Python tutorials to get an introduction to expressions, variables, and basic types, such as strings.
In your case you need to run what's in the string as an actual expression to get the returned face sequence, and build an array of faces from that. To make it more complicated, Abaqus returns their own internal type, Sequence, when you call getByBoundingBox. This type is a pain to work with because you can't create an empty one (at least not that I'm aware of). So dynamically building a list of faces for a set requires some extra attention. In the below code I get the face sequence for each rebar instance, and then add each individual face to my own list of faces. Finally, to create the set, Abaqus is picky about the type on the faces argument. We need to create a part.FaceArray object for Abaqus to be satisfied.
import part
faces = []
for i in rebar_instances:
face_sequence = m.rootAssembly.instances[i].faces.getByBoundingBox(0,0,0,X,Y,Z)
faces += [f for f in face_sequence]
m.rootAssembly.Set(faces=part.FaceArray(faces), name='A')
I am a beginner in python and learning "Learn Python: The Complete Python Programming Course" using Udemy. I downloaded Python 3.8. All things were good until I reached placeholders:
for i in arr.
Print(sen%(“i”))
I just want to use the function of placeholders in python and complete the task by placing multiple names in an array. I cannot type any of this. May I know why?
I am stuck and can't move ahead in the learning of the course.
Try:
for i in arr:
print(sen%(i))
That is, replace arr. to arr:
It is a bit finicky and will throw an error if the statement isn't lined up correctly, if you hit return in the wrong spot, or put a space where there shouldn't be one.
As #Joshua Varghese suggested, first change arr. to arr:
Make sure there is no space in arr:print(sen%(i))
Put your curser just before print(sen%(i)), and hit return. (This will put your print statement on the next line with the correct spacing).
To have your program execute and iterate through the array of names, you need to have your curser on the next line, the line following print(sen%(i)), then hit return.
Note: using python 3.9.2 on Mac OS
I am trying to transfer controls from one rig to another.
I think I have most of it figured out but I'm getting a bit stuck.
I have a function that I am feeding, the duplicated control that I want to attach as well as the list of controls from the original rig that I need to find to move the control to.
My issue is that I keep getting this error:
Error: ValueError: file line 132: More than one object matches name: Index_2_L_ctrl
I searched through everything and I'm pretty sure that there is only one thing named each, but I can't figure out how to list any additional items named the same way. Or better yet to get rid of them.
Here is my function; let me know if anything is unclear I will try to clarify:
def spltString(wtlf, arr):
ndp = wtlf
print ndp
dlb = difflib.get_close_matches(ndp, arr)
fil = dlb[0]
cmds.pointConstraint(ndp, dlb[0])
Try passing in the long names of the controls you want, rather than the short names. That will disambiguate different copies of Index_2_L_ctrl
You can find all of the copies of the control like this:
controls = cmds.ls('Index_2_L_ctrl', long = True)
the results will be object names with the complete hierarchy prepended, like
|skeleton|pelvis|spine1|spine2|chest|r_arm|r_forearm
or whatever. cmds.ls() with the long=True flag will convert short names to long ones for you.
It's a good habit to use long names most of the time precisely because of the problems you're having.
Ok, so I'm writing some python code (I don't write python much, I'm more used to java and C).
Anyway, so I have collection of integer literals I need to store.
(Ideally >10,000 of them, currently I've only got 1000 of them)
I would have liked to be accessing the literals by file IO, or by accessing there source API, but that is disallowed.
And not ontopic anyway.
So I have the literals put into a list:
src=list(0,1,2,2,2,0,1,2,... ,2,1,2,1,1,0,2,1)
#some code that uses the src
But when I try to run the file it comes up with an error because there are more than 255 arguments.
So the constructor is the problem.
How should I do this?
The data is intitally avaiable to me as a space deliminated textfile.
I just searched and replaced and copied it in
If you use [] instead of list(), you won't run into the limit because [] is not a function.
src = [0,1,2,2,2,0,1,2,... ,2,1,2,1,1,0,2,1]
src = [int(value) for value in open('mycsv.csv').read().split(',') if value.strip()]
Or are you not able to save text file in your system?