I'm new to pact and I've understood the concept but having hard time understanding and implementing the code.
Here I'm trying to do a simple pact for get_users from reqres.in.
I believe the first (pact ... code does the mock provider part and I compare that using the pact.json file.
import os
import requests
import pytest
from pact import Consumer, Provider, Format
import unittest
import json
pact = Consumer('Consumer').has_pact_with(Provider('Provider'), port=1234, host_name='localhost')
pact.start_service()
CURR_FILE_PATH = os.path.dirname(os.path.abspath(__file__))
PACT_DIR = os.path.join(CURR_FILE_PATH, '')
PACT_FILE = os.path.join(PACT_DIR, 'pact.json')
#defining class
class GetUsers(unittest.TestCase):
def test_get_board(self):
with open(os.path.join(PACT_DIR, PACT_FILE), 'rb') as path_file:
pact_file_json = json.load(path_file)
print('pact_json')
(pact
.given('Request to send message')
.upon_receiving('a request for response or send message')
.with_request(method = 'GET', path = '/api/users?page=2')
.will_respond_with(status = 200, body = pact_file_json))
with pact:
result = requests.get('http://reqres.in/api/users?page=2')
print('actual response')
self.assertEqual(pact_file_json, result.json())
pact.verify()
ge = GetUsers()
print(ge.test_get_board())
however, when I run the code, I get the following error
the data do not match but I verified it in another code.
Traceback (most recent call last):
File "C:\Users\Desktop\Pact\contract_test.py", line 41, in <module>
print(ge.test_get_board())
File "C:\Users\Desktop\Pact\contract_test.py", line 34, in test_get_board
print(data)
File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pact\pact.py", line 370, in __exit__
self.verify()
File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pact\pact.py", line 269, in verify
assert resp.status_code == 200, resp.text
AssertionError: Actual interactions do not match expected interactions for mock MockService.
says missing requests
Missing requests:
GET https://reqres.in/api/users?page=2
I'm not sure what you're trying to attempt here - why are you opening up a pact.json file in this context? The consumer pact package will automatically serialise any contracts if successful, and you don't seem to be reading the file in anywhere.
The second problem seems to be that some code somewhere, is issuing (or requesting?) the query string page=2 but the code you've shown doesn't make use of any query strings.
Could you have another mock service running somewhere?
Examples
https://docs.pactflow.io/docs/examples/python/consumer/readme
https://github.com/pact-foundation/pact-python/blob/master/examples/e2e/tests/consumer/test_user_consumer.py
Related
Im new to pyflink. Im tryig to write a python program to read data from kafka topic and prints data to stdout. I followed the link Flink Python Datastream API Kafka Producer Sink Serializaion. But i keep seeing NoSuchMethodError due to version mismatch. I have added the flink-sql-kafka-connector available at https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka_2.11/1.13.0/flink-sql-connector-kafka_2.11-1.13.0.jar. Can someone help me in with a proper example to do this? Following is my code
import json
import os
from pyflink.common import SimpleStringSchema
from pyflink.datastream import StreamExecutionEnvironment
from pyflink.datastream.connectors import FlinkKafkaConsumer
from pyflink.common.typeinfo import Types
def my_map(obj):
json_obj = json.loads(json.loads(obj))
return json.dumps(json_obj["name"])
def kafkaread():
env = StreamExecutionEnvironment.get_execution_environment()
env.add_jars("file:///automation/flink/flink-sql-connector-kafka_2.11-1.10.1.jar")
deserialization_schema = SimpleStringSchema()
kafkaSource = FlinkKafkaConsumer(
topics='test',
deserialization_schema=deserialization_schema,
properties={'bootstrap.servers': '10.234.175.22:9092', 'group.id': 'test'}
)
ds = env.add_source(kafkaSource).print()
env.execute('kafkaread')
if __name__ == '__main__':
kafkaread()
But python doesnt recognise the jar file and throws the following error.
Traceback (most recent call last):
File "flinkKafka.py", line 31, in <module>
kafkaread()
File "flinkKafka.py", line 20, in kafkaread
kafkaSource = FlinkKafkaConsumer(
File "/automation/flink/venv/lib/python3.8/site-packages/pyflink/datastream/connectors.py", line 186, in __init__
j_flink_kafka_consumer = _get_kafka_consumer(topics, properties, deserialization_schema,
File "/automation/flink/venv/lib/python3.8/site-packages/pyflink/datastream/connectors.py", line 336, in _get_kafka_consumer
j_flink_kafka_consumer = j_consumer_clz(topics,
File "/automation/flink/venv/lib/python3.8/site-packages/pyflink/util/exceptions.py", line 185, in wrapped_call
raise TypeError(
TypeError: Could not found the Java class 'org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer'. The Java dependencies could be specified via command line argument '--jarfile' or the config option 'pipeline.jars'
What is the correct location to add the jar file?
I see that you downloaded flink-sql-connector-kafka_2.11-1.13.0.jar, but the code loades flink-sql-connector-kafka_2.11-1.10.1.jar.
May be you can have a check
just need to check the path to flink-sql-connector jar
You should add jar file of flink-sql-connector-kafka, it depends on your pyflink and scala version. If versions are true, check your path in add_jars function if the jar package is here.
I've been trying to follow the examples and documentation for the python ad_manager library for the google ads API, but I haven't been able to complete a successful request. I currently have my developer token, client_id, client_secret, and refresh_token in my google ads YAML file, but I'm constantly getting the error "argument should be integer or bytes-like object, not 'str'" when calling the function WaitForReport following the example code below. I was wondering if anyone had any advice on how I could tackle this issue.
import tempfile
# Import appropriate modules from the client library.
from googleads import ad_manager
from googleads import errors
def main(client):
# Initialize a DataDownloader.
report_downloader = client.GetDataDownloader(version='v202111')
# Create report job.
report_job = {
'reportQuery': {
'dimensions': ['COUNTRY_NAME', 'LINE_ITEM_ID', 'LINE_ITEM_NAME'],
'columns': ['UNIQUE_REACH_FREQUENCY', 'UNIQUE_REACH_IMPRESSIONS',
'UNIQUE_REACH'],
'dateRangeType': 'REACH_LIFETIME'
}
}
try:
# Run the report and wait for it to finish.
report_job_id = report_downloader.WaitForReport(report_job)
except errors.AdManagerReportError as e:
print('Failed to generate report. Error was: %s' % e)
# Change to your preferred export format.
export_format = 'CSV_DUMP'
report_file = tempfile.NamedTemporaryFile(suffix='.csv.gz', delete=False)
# Download report data.
report_downloader.DownloadReportToFile(
report_job_id, export_format, report_file)
report_file.close()
# Display results.
print('Report job with id "%s" downloaded to:\n%s' % (
report_job_id, report_file.name))
if __name__ == '__main__':
# Initialize client object.
ad_manager_client = ad_manager.AdManagerClient.LoadFromStorage()
main(ad_manager_client)
Edit:
Below is the stack trace:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/googleads/common.py", line 984, in MakeSoapRequest
return soap_service_method(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zeep/proxy.py", line 46, in __call__
return self._proxy._binding.send(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zeep/wsdl/bindings/soap.py", line 135, in send
return self.process_reply(client, operation_obj, response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zeep/wsdl/bindings/soap.py", line 229, in process_reply
return self.process_error(doc, operation)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zeep/wsdl/bindings/soap.py", line 317, in process_error
raise Fault(
zeep.exceptions.Fault: Unknown fault occured
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "google_ads.py", line 72, in <module>
main(ad_manager_client)
File "google_ads.py", line 33, in main1
report_job_id = report_downloader.WaitForReport(report_job)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/googleads/ad_manager.py", line 784, in WaitForReport
report_job_id = service.runReportJob(report_job)['id']
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/googleads/common.py", line 989, in MakeSoapRequest
underlying_exception = e.detail.find(
TypeError: argument should be integer or bytes-like object, not 'str'
In your YAML file, do you have your account number in quotes? (either single or double?)
Additionally, I would highly recommend not going with this API if you have the option. It will be sunset in April and will no longer work. The newer google ads API (as opposed to the AdWords API) is available, stable and much easier to work with. The ad manager examples are good too.
The problem seems to be that zeep raises a WebFault which includes the returned XML response as a string in zeep.Fault.detail.
Somewhat counter-intuitive, this attribute is not a string, but a bytes sequence because zeep.wsdl.utils.etree_to_string calls etree.tostring() with encoding="utf-8" instead of encoding="unicode"—the latter would make sure it's a proper string.
googleads then tries to look for specific error strings inside the XML using find(), but even though find() is defined both on str and bytes, the type of the substring to look for needs to align.
Thus, in
underlying_exception = e.detail.find(
'{%s}ApiExceptionFault' % self._GetBindingNamespace())
bytes.find() is called with a str argument, causing the ValueError you experience.
I'd argue that zeep.wsdl.utils.etree_to_string() should be adjusted to actually return a str instead of bytes. You could try opening an issue on Zeep's Github repository.
I am trying to prepare a VM Resizing Recommendations Report using a Python 3.7 script. My code is as follows:
import datetime
import logging
from google.cloud import bigquery
from google.cloud import recommender
from google.cloud.exceptions import NotFound
from googleapiclient import discovery
def main(event, context):
client = recommender.RecommenderClient()
recommender_type = "google.compute.instance.MachineTypeRecommender"
projects=list_projects() #This gives list of projects
# I hard-code the zones below:
zones = ["us-east1-b","us-east1-c","us-east1-d","us-east4-c","us-east4-b","us-east4-a"
,"us-central1-c","us-central1-a","us-central1-f","us-central1-b"
,"us-west1-b","us-west1-c","us-west1-a"]
for zone in zones:
parent = client.recommender_path("my-project", zone, recommender_type)
for element in client.list_recommendations(parent): #In this line I am getting this error and these are logs
****Parent**** projects/my-project/locations/us-east1-b/recommenders/google.compute.instance.MachineTypeRecommender
Traceback (most recent call last):
File "main.py", line 187, in main
x=list(client.list_recommendations(parent))
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/google/cloud/recommender_v1/services/recommender/client.py", line 734, in list_recommendations
request = recommender_service.ListRecommendationsRequest(request)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/proto/message.py", line 441, in __init__
raise TypeError(
TypeError: Invalid constructor input for ListRecommendationsRequest: 'projects/my-project/locations/us-east1-b/recommenders/google.compute.instance.MachineTypeRecommender'
During handling of the above exception, another exception occurred:
"projects/my-project/locations/us-east1-b/recommenders/google.compute.instance.MachineTypeRecommender" is the parameter being passed to the list_recommendations function. I am not sure what is wrong in the constructor as I am getting this: Invalid constructor input for ListRecommendationsRequest
I am new to this Google api. What can I try next?
I have checked the code on my end an it seems to be a syntax error when calling the method. As per the library documentation the first parameter should be a request object or None and the parent parameter has to be passed by name. Changing the line as follows I didn't have the error:
client.list_recommendations(parent=parent)
I have a fairly simple use-case but i'm not understanding the error message i'm receiving.
I'm using the requests and pyral modules, pyral (http://pyral.readthedocs.io/en/latest/interface.html#) is really just a wrapper for Rally's Restful api. My goal is to get a file (attachment) from a Rally (a CA product) UserStory and store it to a local file system.
For context, here is my environment setup (authenticate to Rally and create an object). I've obviously removed authentication information.
from pyral import Rally, rallyWorkset
options = [arg for arg in sys.argv[1:] if arg.startswith('--')]
args = [arg for arg in sys.argv[1:] if arg not in options]
server, user, password, apikey, workspace, project = rallyWorkset(options)
rally = Rally(server='rally1.rallydev.com',
user='**********', password='***********',
apikey="**************",
workspace='**************', project='**************',
server_ping=False)
After that I get a response object for just one user story (see the query for US845), i do this just to simplify the problem.
r = rally.get('UserStory', fetch = True, projectScopeDown=True, query = 'FormattedID = US845')
and then I use the built-in iterator to get the user story from the RallyRESTResponse object.
us = r.next()
from there it feels like I should be able to easily use the getAttachment() method that accepts a artifact (us) and filename (name of an attachment). I'm able to use getAttachmentNames(us) to return a list of attachment names. The issue arrises when i try something like
attachment_names = rally.getAttachmentNames(us) #get attachments for this UserStory
attachment_file = rally.getAttachment(us, attachment_names[0]) #Try to get the first attachment
returns an error like this
Traceback (most recent call last):
File "<ipython-input-81-a4a342a59c5a>", line 1, in <module>
attachment_file = rally.getAttachment(us, attachment_names[0])
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1700, in getAttachment
att.Content = base64.decodebytes(att_content.Content) # maybe further txfm to Unicode ?
File "C:\Miniconda3\lib\base64.py", line 552, in decodebytes
_input_type_check(s)
File "C:\Miniconda3\lib\base64.py", line 520, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
I receive a similar error if i try to use
test_obj = rally.getAttachments(us)
Which returns an error like this:
Traceback (most recent call last):
File "<ipython-input-82-06a8cd525177>", line 1, in <module>
rally.getAttachments(us)
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1721, in getAttachments
attachments = [self.getAttachment(artifact, attachment_name) for attachment_name in attachment_names]
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1721, in <listcomp>
attachments = [self.getAttachment(artifact, attachment_name) for attachment_name in attachment_names]
File "C:\Miniconda3\lib\site-packages\pyral\restapi.py", line 1700, in getAttachment
att.Content = base64.decodebytes(att_content.Content) # maybe further txfm to Unicode ?
File "C:\Miniconda3\lib\base64.py", line 552, in decodebytes
_input_type_check(s)
File "C:\Miniconda3\lib\base64.py", line 520, in _input_type_check
raise TypeError(msg) from err
TypeError: expected bytes-like object, not str
It seems that i'm fundamentally misunderstanding the parameters that this method requires? Has anyone been able to do this successfully before? For what it's worth i have no issues using the addAttachment() method with a workflow similar to the above. I've tried converting the filename (string) with the bytes() method to utf-8 but that didn't help.
I've also looked at this example in the pyral source, but i receive exactly the same error when trying to execute that.
https://github.com/klehman-rally/pyral/blob/master/examples/get_attachments.py
It looks like the issue in restapi.py script - there is no decodebytes method in base64 library:
att.Content = base64.decodebytes(att_content.Content)
All available methods are described at:
RFC 3548: Base16, Base32, Base64 Data Encodings
So, workaround is to replace decodebytes by base64.b64decode in restapi.py. At least, it works me.
E.g. location at Mac OS X:
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyral/restapi.py
I have used the below code to get all attachment since getAttachments is not working as expected. it will create a file in the current dir with the same name.
import sys
import string
import base64
from pyral import rallyWorkset, Rally,RallyRESTResponse
rally = Rally(server, user=USER_NAME, password=PASSWORD, workspace=workspace, project=project)
criterion = 'FormattedID = US57844'
response = rally.get('HierarchicalRequirement', query=criterion, order="FormattedID",pagesize=200, limit=400, projectScopeDown=True)
artifact = response.next()
context, augments = rally.contextHelper.identifyContext()
for att in artifact.Attachments:
resp = rally._getResourceByOID(context, 'AttachmentContent', att.Content.oid, project=None)
if resp.status_code not in [200, 201, 202]:
break
res = RallyRESTResponse(rally.session, context, "AttachmentContent.x", resp, "full", 1)
if res.errors or res.resultCount != 1:
print("breaking the for loop")
att_content = res.next()
cont = att_content.Content
x = base64.b64decode(cont)
output = open(att.Name, 'wb')
output.write(x)
I am using simple Python script to fetch example data from Solr using Pysolr. First I created my core using the following
[user#user solr-7.1.0]$ ./bin/solr create -c json_db
WARNING: Using _default configset. Data driven schema functionality is enabled by default, which is
NOT RECOMMENDED for production use.
To turn it off:
curl http://localhost:8983/solr/json_db/config -d '{"set-user-property": {"update.autoCreateFields":"false"}}'
Created new core 'json_db'
[user#user solr-7.1.0]$ ./bin/post -c json_db example/exampledocs/*.json
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/json_db/update...
Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file books.json (application/json) to [base]/json/docs
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/json_db/update...
Time spent: 0:00:00.398
After creating the core I ran simple python script to fetch data
from pysolr import Solr
conn = Solr('http://localhost:8983/solr/json_db/')
results = conn.search('*:*')
I am getting this error
Traceback (most recent call last):
File "/home/user/PycharmProjects/APP/application/solr_test.py", line 4, in <module>
results = conn.search({'*:*'})
File "/home/user/PycharmProjects/APP/venv/lib/python3.5/site-packages/pysolr.py", line 723, in search
response = self._select(params, handler=search_handler)
File "/home/user/PycharmProjects/APP/venv/lib/python3.5/site-packages/pysolr.py", line 421, in _select
return self._send_request('get', path)
File "/home/user/PycharmProjects/APP/venv/lib/python3.5/site-packages/pysolr.py", line 396, in _send_request
raise SolrError(error_message % (resp.status_code, solr_message))
pysolr.SolrError: Solr responded with an error (HTTP 404): [Reason: Error 404 Not Found]
But when I try to run the query directly from solr I got results like the following
Can somebody guide me what I am doing wrong here ? Thanks
You can just run the script below to fetch the result without using pysolr library.
#! /usr/bin/python
import urllib
import json as simplejson
import pprint
import sys
url = 'give the url here'
wt = "wt=json"
connection = urllib.urlopen(url)
if wt == "wt=json":
response = simplejson.load(connection)
else:
response = eval(connection.read())
print "Number of hits: " + str(response['response']['numFound'])
pprint.pprint(response['response']['docs'])