I am using Python, Docker, and Locust to load test a service I am working on. I have some test data I need to partition uniquely between how ever many docker containers I spin up dynamically. Locust gives all of it's worker containers a name styled as "worker_N". How can I get the worker name assigned to the container by checking inside the python script itself? I am using docker compose to spin up the containers, so I can't use the similar solution provided here.
Depending on exactly what it is you're trying to do, you could set a hostname or maybe an alias for each worker. (Locust uses hostname + a GUID for the name of the workers.) If you need to access each worker somehow, you could set the hostname to include an IP address. Just an idea.
To address your original question more directly, there are also other ways to access hostname, such as in Python code (like Locust does).
One last option would be to use the master do push the data you need to each worker using Locust's messages feature. Then you only need to ensure the master has access to whatever data you want distributed to the workers and it can divvy it up to however many workers are connected.
Related
Where can I get a list of all the docker status types? e.g. Up, Exited, Created.
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f0771636c8ab registry:2 "/entrypoint.sh /etc…" 25 hours ago Up 3 hours 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp registry
This next part may be unrelated since it could be a completely different "status". but in the docker python api I've also seen status values of the following. I looked through the Python docker code and it doesn't appear to include these string so I think they're originating within docker itself and not the python api.
preparing
downloading
pushing
restarting
running
waiting
verifying checksum
etc.
In the Docker HTTP API, the Inspect a Container API call (GET /containers/{id}/json) includes a Stats field with OpenAPI type ContainerState. That contains a field Status. Its possible values are "created" "running" "paused" "restarting" "removing" "exited" "dead"
The higher-level Docker SDKs and CLI tools all ultimately wrap this API, so any container status from docker-py or docker ps will be derived from one of these values. The Up 3 hours output, for example, looks like a combination of a running state and a calculated container uptime. The list you quote largely doesn't look like container statuses ("push" is not a valid action on a container) and it might go with some other object.
I have created 15 servers which contain the same machine learning program. However, each server has different arguments at the runtime. It is determined by the hostname. Each server also has a copy of a 5 1gb pkl files which contain training data.
So for example right now, I have created 15 servers in the cloud with the following names.
ml-node-1
ml-node-2
ml-node-3
..
ml-node-15
So when my program runs on ml-node-1 it looks like this, python3 mlscript.py $HOSTNAME.csv and it will run python3 mlscript.py ml-node-1.csv. Each server will run the script which is meant for its hostname.
My problem is that I have to create 15 copies of the 5gb pkl data in each server before they are run. I find this very inefficient and costly therefore I am looking up Kubernetes as a solution. From the documentation, I can see that containers within a pod can share a persistent volume. This way I might be able to mitigate copying the 5gb pkl data 15 times.
However, I am trying to figure out the naming of the servers/containers. I figure that I would need 1 pod with a shared persistent volume and 15 containers. According to what I can understand from the documentation, https://kubernetes.io/docs/concepts/containers/container-environment/, all the containers within the pod will share the same hostname.
How do I differentiate them and give them specific hostnames so that I can still have a different argument running within each container? My docker images are the standard debian image with my machine learning script.
Rather than relying on custom hostnames for a scalable multi-host environment like Kubernetes (which is not designed to allow for this), as a more feasible solution (suggested by MatsLindh) you could write some code in your mlscript.py that generates a unique random key on startup and then watches an input directory for changes in a continuous loop: This directory would contain the available files to be picked up by the different containers, and the script would rename a given file with the generated key when it assigns it to the running server instance in the container, with "unassigned" files (not containing a key with the same format in the name) being considered available for other instances.
This would allow you to scale the workload up or down from Kubernetes as well as have several files processed by the same container instance.
I am using Pyro4 in a distributed system. My problem is that after two networks connect there can be many nameserver. It is ok for me but if any of them is empty (has no objects registered) my network can fail .
This is because I am implementing chord and I need to know if there is any chord node running already (if the empty nameserver responds then two chord rings will be created.
Is there any way to kill a nameserver if it is empty?
My code is in python3. Thanks
Pyro's name server assumes it is running all the time: you never know if there will be someone a bit later that wants to register a new name. If you really need it to auto shutdown, you'll have to either wrap the default name server in some custom code, or use some form of watchdog process that kills the name server if it detects that it has 0 entries.
You can get the number of entries via a regular Pyro call to the nameserver: ns.count(). For obvious reasons, it's not possible to remotely shutdown the nameserver via such a Pyro API call, hence the need of a watch dog process.
I have a use case, where I have an EC2 instance with Fedora Linux and some applications running. When this instance fails, I have to spin up a new instance with the same OS and install the applications. I am trying to do in Ansible (and Python), I'm a complete novice and have no idea how to do it.
For my applications, I have a variable (a structure) that tells me how many of each type of server I need in each of three subnets. The environment creation playbook loops through that structure and builds however many are needed to fill the requirements. So if I need five (5) and only have three (3), it builds two (2). I use exact_count for that in the ec2 module.
So if one fails, I can delete that instance and re-run my create playbook, which will also re-write all the configuration files on the other servers that they use to communicate with each other. For instance, if I delete a JBoss server create a new one to replace it, the load balancer has to know about it.
Good practise here would be to have a base image that covers what you need, use that as a feeder for an AMI, and then plug it into an Auto-scaling group. As part of the auto-scaling group, you can use user-data to load specific updates/etc onto the instance at boot time.
Autoscale group min 1 max 1 will do exactly what you want, if you can configure it the above way.
I'm doing an embarrassingly parallel operation on Amazon Web Services, in which I'm spinning up a large number of EC2 instance that all have slightly different scripts to run on startup. Currently, I'm starting up each instance individually within a for loop like so (I'm using the Python boto package to talk to AWS):
for parameters in parameter_list:
#Create this instance's startup script
user_data = startup_script%parameters
#Run this instance
reservation = ec2.run_instances(ami,
key_name=key_name,
security_groups=group_name,
instance_type=instance_type,
user_data=user_data)
However, this takes too long. ec2.run_instances allows one to start many instances at once, using the max_count keyword. I would like to create many instance simultaneously passing each their own unique startup script (user_data). Is there any way to do this? One cannot just pass a list of scripts to user_data.
One option would be to pass the same startup script, but have the script reference another peice of data associated with that instance. EC2's tag system could work, but I don't know of a way to assign tags in a similarly parallel fashion. Is there any kind of instance-specific data I can assign to a set of instances in parallel?
AFAIK, there is no simple solution. How about using Simple Queue Service(SQS)?
Add start-up scripts (aka user-data) to SQS
write user-data as
read a start-up script from SQS and run it
If your script is upper than 256k, you do not add it to SQS directly. So, try this procedure.
Add start-up scripts (aka user-data) to S3
Add the S3 url of the script to SQS
write user-data as
read a url from SQS
download the script from S3
run it
Sorry, It's very complicated. Hope this helps.
Simple. Fork just before you initialize each node.
newPid = os.fork()
if newPid == 0:
is_master = False
# Create the instance
...blah blah blah...
else:
logging.info( 'Launched host %s ...' % hostname )