Develop Reference

Read paths in csv and open in unix

I need to read paths in a csv and go to these file paths in unix. I am wondering if there is any way to do that using unix or python commands. I am unsure of how to proceed and I am not able to find much resources in the net either.
The number of rows in the excel.csv is a lot and similar to below. I need to open the excel.csv and then read the first line and go to this file path. Once this file is opened using the file path, I need to be able to read the file and extract out certain information. I tried using python for this but I am unable to find much information and I am wondering if I can use unix commands to solve this. I am clueless on how to proceed for this one so I would appreciate any reference or help using either python or unix commands. Thank you!
/folder/file1
/folder/file2
/folder/file3

It shouldn't be very difficult to do this in Python, as reading csv files is part of the standard library. Your code could look something like this:
with open('data.csv', newline='') as fh:
# depending if the first row describes the header or not,
# you can also use the simple csv.reader here
for row in csv.DictReader(fh, strict=True):
# again, if you use the simple csv.reader, you'll have to access
# the column via index instead
file_path = row['path']
with open(file_path, 'r') as fh2:
data = fh2.read()
# do something with data

Pymongo - store PDF data as Binary

I am working on a project where I want to store PDF data within a Mongo collection. I realize this has been asked around the community, but I am struggling with the result. My code is able to insert a document into the collection, but stalls because it enters a blank binary object.
import base64, os
import pymongo
import bson
from pathlib import Path
file_used = 'sample.pdf'
my_file = Path(file_used)
if my_file.is_file():
print("File Exists") #prints successfully
with open(file_used, 'rb') as fout:
string = base64.b64encode(fout.read())
collection.insert_one({"filename":file_used,"pdf_data":string})
Document in MDB Compass:
{
"_id":{"$oid":"60583bfdffea362641c50944"},
"filename":"sample.pdf",
"pdf_data":{"$binary":"","$type":"0"}
}
My python file is also located in the same directory as my sample PDF file.
Execution: python3 test.py
How can I debug my code further? Thank you in advance for your help?

Looks like something was wrong with the PDF file. It worked once I used another PDF file as my test case.

How to download file from link like this?

I want to ask how to download file using Python from a link like this, I crawled through the stack for a while and didn't find anything that works.
I got a link to a file, something like this:
https://w3.google.com/tools/cio/forms/anon/org/contentload?content=https://w3.ibm.com/tools/cio/forms/secure/org/data/f48f2294-495b-48f5-8d4e-e418f4b25a48/F_Form1/attachment/bba4ddfd-837d-47a6-87ef-2114f6b3da08 (link doesn't work, just showing you how it should look)
And after clicking on it it opens a browser and starts opening file:
I don't know how the file will be named or what format file will have, I only have a URL that links to file like this image up.
I tried this:
def Download(link):
r = requests.get(link)
with open('filename.docx', 'wb') as f:
f.write(r.content)
But this definitely doesn't work, as you can see I manually put the name of the file because it desperate but it doesn't work either, it makes file but only 1kb size and nothing in it.
I don't know how to code it to automatically download it from links like this? Can you help?

use urlretrieve from urllib. See here

You can use urllib.request.urlretrieve to get the contents of the file.
Example:
import urllib.request
with open('filename.docx', 'wb') as f:
f.write(urllib.request.urlretrieve("https://w3.google.com/tools/cio/forms/anon/org/contentload?content=https://w3.ibm.com/tools/cio/forms/secure/org/data/f48f2294-495b-48f5-8d4e-e418f4b25a48/F_Form1/attachment/bba4ddfd-837d-47a6-87ef-2114f6b3da08"))

Run python zip file from memory at runtime?

I am trying to run a python zip file which is retrieved using requests.get. The zip file has several directories of python files in addition to the __main__.py, so in the interest of easily sending it as a single file, I am zipping it.
I know the file is being sent correctly, as I can save it to a file and then run it, however, I want to execute it without writing it to storage.
The working part is more or less as follows:
import requests
response = requests.get("http://myurl.com/get_zip")
I can write the zip to file using
f = open("myapp.zip","wb")
f.write(response.content)
f.close()
and manually run it from command line. However, I want something more like
exec(response.content)
This doesn't work since it's still compressed, but you get the idea.
I am also open to ideas that replace the zip with some other format of sending the code over internet, if it's easier to execute it from memory.

A possible solution is this
import io
import requests
from zipfile import ZipFile
response = requests.get("http://myurl.com/get_zip")
# Read the contents of the zip into a bytes object.
binary_zip = io.BytesIO(response.content)
# Convert the bytes object into a ZipFile.
zip_file = ZipFile(binary_zip, "r")
# Iterate over all files in the zip (folders should be also ok).
for script_file in zip_file.namelist():
exec(zip_file.read(script_file))
But it is a bit convoluted and probably can be improved.

Updated version of yaml file is not getting read

I am reading a yaml file like so in Python3:
def get_file():
file_name = "file.yml"
properties_stream = open(file_name, 'r')
properties_file = yaml.safe_load(properties_stream)
properties_stream.close()
print(properties_file)
return properties_file
When I update file.yml by adding keys or deleting all the contents of the file, the "print(properties_file)" statement is still retaining the contents of the yaml file the first time I ran this code.
Any ideas why this might be happening?

Anthon was correct. Turned out I had two folders with the same name in my project directory in a different subdirectory and it was reading from the wrong file.