Develop Reference

How do I specify a start date using time at a particular time of the day with yfinance

Please am trying to get the stock date at a specify time of the date adding the time of which i want the stock data to be gotten from as well to so far i did something like this but i keep getting error saying data - F: Data doesn't exist for startDate = 1666286770, endDate = 1666287074
this is my code below
def watchlist():
timezone = pytz.timezone('US/Eastern')
print(type(timezone))
aware = dt.datetime.now(timezone).time()
print(aware)
global pastTime
pastTime = dt.datetime.now(timezone) - dt.timedelta(minutes=5) # time of 5minutes ago
print(pastTime)
for x in ticks:
toStr = str(x)
syb = yf.Ticker(toStr)
data = pd.DataFrame(syb.history(interval="1m",period='1d',))
data2 = pd.DataFrame(syb.history(interval="1m",period='1d',start=pastTime))
if data['Open'].sum() < data2['Open'].sum():
print(data['Open'].sum())
print(data2['Open'].sum())
print('Watch stock')
else:
print(toStr, 'Proceed to sell with robinhood')
watchlist()
Screen shot of issue

Filter icalendar events based on a time frame breaks because of datetime comparison

I am trying to pull events from an ics file within the next month but making my own datetime and comparing to the datetime in the ics file doesn't seem to be working and is giving the error TypeError: can't compare offset-naive and offset-aware datetimes
I tried the answers found here but still get the same error. Below is the code I am using.
def read_from_cal():
g = open('calendar.ics', 'rb')
gcal = Calendar.from_ical(g.read())
year = datetime.now().year
month = datetime.now().month
day = datetime.now().day
hour = datetime.now().strftime("%H")
minute = datetime.now().strftime("%M")
next_month = datetime(int(year), int(month)+1, int(day), int(hour), int(minute), 0, tzinfo=pytz.UTC)
#next_month = next_month.replace(tzinfo=pytz.UTC)
for component in gcal.walk():
if component.name == "VEVENT":
# time printed out in format:
# year-month-day hour:min:sec
summ = component.get('summary')
start = component.get('dtstart').dt
end = component.get('dtend').dt
if now <= start <= next_month:
print("Worked")
print(summ, start, end)
I've tried both with using replace to change my time to utc and just putting it in the next_month variable itself and they both give the same error above.
I've also tried this and this to no avail.

I have tried with a .ics file generated here, so could not be the same problem, but in some cases start is a datetime.datetime and in other cases is a datetime.date.
This solution worked with my .ics file
from icalendar import Calendar
from datetime import datetime
from dateutil.relativedelta import relativedelta
def read_from_cal():
g = open('example.ics', 'rb')
gcal = Calendar.from_ical(g.read())
today = datetime.now()
next_month = today + relativedelta(months=1)
for component in gcal.walk():
if component.name == "VEVENT":
summ = component.get('summary')
start = component.get('dtstart').dt
end = component.get('dtend').dt
if isinstance(start, datetime):
start = start.replace(tzinfo=None)
if start <= next_month:
print("Worked (datetime)")
print(summ, start, end)
else:
# some events are stored as a date
if start <= next_month.date():
print("Worked (date)")
print(summ, start, end)
read_from_cal()

Getting BTC historical data by Kraken API

I'm trying to getting data from kraken exchange by the krakenex API. But i'm facing several problems, 'cause, I want getting the data in a range time bigger than the alllowed by the API.
The API only allows getting a dataframe with 720 rows, so 'cause that I need to do a loop while to getting more data and concat in another dataframe.
I've already read other topics about it, but I'm still not reaching good results.
import krakenex
import time
import krakenex
import pandas as pd
from pykrakenapi import KrakenAPI
from datetime import datetime
k = krakenex.API()
start = '28/01/2021 00:00:00'
start = datetime.strptime(start, "%d/%m/%Y %H:%M:%S")
start = int(time.mktime(start.timetuple()))
stop = '03/02/2021 00:00:00'
stop = datetime.strptime(stop, "%d/%m/%Y %H:%M:%S")
stop = int(time.mktime(stop.timetuple()))
prices = pd.DataFrame()
while start < stop:
time.sleep(5)
data = k.query_public('OHLC', {'pair':'XXBTZUSD', 'interval':1, 'since':start})
df = pd.DataFrame( data['result']['XXBTZUSD'])
daily_prices = df[0].to_list()
start = int(daily_prices[0])
prices = pd.concat([precos , df])

For weeks I have been working on a script that does exactly that. In my case I collect all pairs with BTC and ETH but you can use the script with any pair. To do this I used the REST API and defined some functions that automate everything. I download the data with 1 minute timeframe but it can be used for any timeframe.
First I defined a function that downloads the data in full or from a specific date, it's necessary because at the first run it will download all the data and then it will download only the new data. The parameter 'interval' defines the number of minutes of the timeframe while 'since' defines the beginning of the data to download.
def get_ohlc (pair, interval=1, since='last'):
endpoint = 'https://api.kraken.com/0/public/OHLC'
payLoad = {
'pair': pair,
'interval': interval,
'since' : since
}
response = requests.get(endpoint, payLoad)
data = response.json()
OHLC = data['result'][pair]
data = pd.DataFrame.from_records(OHLC, columns=['Time', 'Open', 'High', 'Low', 'Close', 'vwap', 'volume', 'count'])
data['Time'] = pd.to_datetime(data['Time'], unit='s')
data.set_index('Time',inplace=True)
data = data.drop(['vwap', 'volume', 'count'], axis=1)
data['Open'] = data.Open.astype(float)
data['High'] = data.High.astype(float)
data['Low'] = data.Low.astype(float)
data['Close'] = data.Close.astype(float)
return data
Then I defined a function to load the .json file that was saved into memory. The function returns the dataframe with the old data and a timestamp that indicates from where to download the new data. I also created a function for calculate the timestamp.
def load_data(pair, path):
data = pd.read_json(path + pair + '.json' , orient='split')
tmp = data.tail(1).index
tmp = tmp.strftime('%Y-%m-%d %H:%M:%S')
dt = str_to_datetime(tmp[0])
ts = dt.timestamp()
return data, ts
def str_to_datetime(datestr):
Y = int(datestr[0:4])
M = int(datestr[5:7])
D = int(datestr[8:10])
H = int(datestr[11:13])
m = int(datestr[14:16])
return datetime.datetime(Y, M, D, H, m, 0, tzinfo=tz.gettz("Etc/GMT"))
Now your main should be something like:
from countdown import countdown
import pandas as pd
import datetime
import os
path = os.getcwd() + '/historical_data/'
pair = 'XBTUSD'
while True:
if os.path.exists(path + pair + '.json') == False:
data = get_ohlc(pair, 1) # 1 minute timeframe
data.to_json(path + pair + '.json', orient='split')
else:
data1, ts = load_data(pair, path)
data2 = get_ohlc(pair, 1, ts)
data3 = pd.concat([data1, data2])
data3.drop(data3.tail(1).index,inplace=True) # delete last record because it's not ended
data3.to_json(path + pair + '.json', orient='split')
countdown(60) # update every hour
I delete the last record because when you download it it's not ended so we will download at the next update. I haven't tested if it works because I took pieces of code from my program, if it doesn't work let me know and I'll fix it.

Retrieving Dates more than Current Time in From Django Database

I am struggling to understand how the date queries work in Django as I am storing a database with train times. I want to get times that are greater than the current time.
The query looks like this, but returns zero results:
latestdepartures = LatestDepartures.objects.filter(station=startstation,earliest__gte=timezone.now().astimezone(pytz.utc))
My database has the entry below for example.
When I run the query, I get the results below (first line is print(timezone.now().astimezone(pytz.utc)):
2020-08-01 15:49:06.610055+00:00
<QuerySet []>
The code which adds the data to the database looks like:
def convert_date_time(o):
if isinstance(o, datetime):
return o.__str__()
def updateservices(stationname,destination):
now = datetime.now()
# dd/mm/YY H:M:S
datenow = now.strftime("%d/%m/%Y")
board = DARWIN_SESH.get_station_board(stationname)
stationdict = dict()
stationdict['from'] = stationname
stationdict['name'] = board.location_name
stationdict['servicelist']=[]
services = board.train_services
for s in services:
traindict = dict()
service_details = DARWIN_SESH.get_service_details(s.service_id)
traindict['departuretime'] = datetime.strptime(datenow + " " + service_details.std,'%m/%d/%Y %H:%M').astimezone(pytz.utc)
traindict['callingpoints'] = []
callingpoints = service_details.subsequent_calling_points
for c in callingpoints:
if c.crs == destination:
callingpointdict = dict()
callingpointdict['code'] = c.crs
callingpointdict['name'] = c.location_name
callingpointdict['arrivaltime'] = datetime.strptime(datenow + " " + c.st,'%m/%d/%Y %H:%M').astimezone(pytz.utc)
traindict['callingpoints'].append(callingpointdict)
if len(traindict['callingpoints']) > 0:
stationdict['servicelist'].append(traindict)
#For getting the minimum departure
departures = [s['departuretime'] for s in stationdict['servicelist']]
#Store the train departure object in the database
stationdata = json.dumps(stationdict, default=convert_date_time)
LatestDepartures.objects.create(
station = stationname,
earliest = min(departures),
services = stationdata
)
return stationdata
servicedetails.std will be a time represented in 24hours in string format, for example "17:00".
Can anyone help, I am not sure if I am meant to change the date format somewhere or if it is to do with the way the datetime object is created by adding the time.
UPDATE:
Now storing the date in a different format as '%d/%m/%Y %H:%M':
Now I get dates that are greater than, but once the time current time has exceeded the earliest in the database, the query still returns results. Example output is:
2020-08-01 17:31:21.909052+00:00
print(timezone.now().astimezone(pytz.utc))
2020-08-01 18:03:00+00:00 - Time in database

Mutiple API with different variable in URL

I am learning Python and had a question regarding for and if loops. This is my scenario:
I have an endpoint that i make API-call with request.get
I need to retrieve all the historic data
I have a start_date (2017-06-17)
So i need to make multiple API-call because they have a limit of 60-days period. So i made my code like this:
date = datetime.strptime("2017-06-17", "%Y-%m-%d") # Start Date
current_date = date.date() # timedelta need date object so i make it a date object
days_after = (current_date+timedelta(days=60)).isoformat() # days_after is set to 60-days because limit in API
date_string = current_date.strftime('%Y-%m-%d') # made to string again since API need string not date object
So this is how i make the dates for 60 days period. Starting from 2017-06-17 and 60-days ahead.
This is how i make the API-request:
response = requests.get("https://reporting-api/campaign?token=xxxxxxxxxx&format=json&fromDate="+date_string+"&toDate="+days_after)
response_data = response.json() # Added this because i am writing temprorary to a JSON file
This is how i write to JSON file:
if response_data:
print("WE GOT DATA") # Debugging
data = response.json() # This is duplicate?
with open('data.json', 'w') as f: # Open my data.json file as write
json.dump(data, f) # dumps my json-data from API to the file
else:
print("NO DATA") # Debugging if no data / response. Should make a skip statement here
So my question is how can i proceed with my code so that every time i make a API-call starting from 2017-06-17 the date date_string and days_after should go 60 days forward for each API-call and append those data to data.json. I would maybe need some for loops or something?
Please note i have been using Python for 3 days now, be gentle.
Thanks!

You could use a while loop that changes the start and end date until a specified condition is met. Also, you can append the response to a file for every run. the example below I used the date of "today":
import os
from datetime import datetime, timedelta
x = 0
y = 60
date = datetime.strptime("2017-06-17", "%Y-%m-%d")
current_date = date.date()
date_start = current_date+timedelta(days=x)
while date_start < datetime.now().date():
date_start = current_date+timedelta(days=x)
days_after = current_date+timedelta(days=y)
x = x + 60
y = y + 60
response = requests.get("https://reporting-api/campaign?token=xxxxxxxxxx&format=json&fromDate="+date_start.isoformat() +"&toDate="+days_after.isoformat())
response_data = response.json()
if response_data:
print("WE GOT DATA")
data = response.json()
#create a file if not exists or append new data to it.
if os.path.exists('data.json'):
append_write = 'a' # append if already exists
else:
append_write = 'w' # make a new file if not
with open('data.json', append_write) as f:
json.dump(data, f)
else:
print("NO DATA")
Basically, on every run the time of start and end is increased by 60 days and appended to the data.json file.