I'm trying to use tablesnap to make backups but without success. I'm using Ubuntu 12.04 and after trying the installation of tablesnap as it is described in github, I'm not able to do it. I guess this is due to fact that the package is for Maverick, so I have tried to copy the code and execute it but again without success. It always display the message "INFO Starting up" and seems nothing happen.
I'm sure the problem is my ignorance but, could you help me? Do you know about any document or example of installing and using for backup and recovery?
UPDATE:
The problem was me. Tablesnap was working but there was no IN_MOVED_TO event. So, now, what I'm trying to do is to backup a complete keyspace. I have tried with the "-B" option of tablesnap but still nothing is uploaded to S3. Any idea?
I'm sure the problem is my ignorance of linux, python and cassandra, but I haven't found enough information to make it work or a step by step document
Being blunt here: yes. You've got the answer to your own question. It's complicated to get used to all of that at once, but a step-by-step document won't help you a bit. Really. You need to be familiar with what you're doing, or else you won't be able to do something useful.
To compare: Installing cassandra is like buying a dentist's chair. Even with a very precise step-by-step information on how to set it up and how to place a patient on it, you'll be a terrible terrible threat to your patient's teeth if you have no education as a dentist before.
Cassandra is a mighty tool for large, ditributed systems. Someone who develops for that or even just administrates that needs to have very solid understanding of how to work with his computer in the environment that cassandra runs in. Get yourself used to linux. Then read a lot about cassandra. Then that project is on your level, and you will have success!
Ok, what I was looking for is very easy. Here is what I have done to make a complete backup of my keyspaces:
python tablesnap -k MY_AWS_KEY -s MY_AWS_SECRET -B my_s3_bucket /opt/cassandra/data/my_keyspace/*
Just replace /opt/cassandra/data/ with the path to your keyspace and that's all. As simple as this is what I was asking for, so I leave it here in case someone finds it useful.
Related
There probably isn't one "right answer" to this question. I'm interested in thoughts and opinions.
We have a couple hundred RHEL7/Centos7/Rocky8 nodes. Many of them have python modules installed via pip/pip3.
I've been searching for a best practices on routine/monthly patching these modules...so I far haven't found any. Obviously things installed with rpm/yum/dnf are pretty easy to deal with.
From the pip man page:
pip install --upgrade SomePackage
Great!
But how do you update all of them?
Sure. It is possible to do a "pip list/freeze" pipe that to awk...etc..
Surely, there's a better way. Ideally, one that captures things like "boto3 V1.2 replaced with boto3 V1.3"
Right now it feels like I'm the only one thinking about this. Maybe I am and it is stupid. I'm ok with that response as well (but please tell me why).
A common solution is to deploy the application code inside a Docker container - the container image contains its own version of Python and all the dependency modules, so you don't have to update each module on all the host machines individually. It also means that the combination of OS, Python and modules that you deploy can be tested and then "frozen" into an immutable image which is then deployed the same everywhere.
Right now it feels like I'm the only one thinking about this.
I realise the above answer is probably not helpful in your situation as you already have a fairly large system deployed... but it might help to explain why not many people are developing solutions to your problem!
I've installed a package called 'prince' to use MCA (multiple correspondance analysis).
I had an issue while executing the code and I've noticed that someone else posted that issue on github and there was a merge pull request to solve that issue.
Since the installation doesn't take that into account, I tried to copy the new script inplace of the previous one and run the code but now I get this error 'attempted relative import beyond top-level package' from the first import which wasn't the case before even if the import was there too.
So I guess when you modify the script even a little bit there's something particular that happen's ? Maybe it's a minor thing and I could still get away with it but I don't quite understand the mechanics here, could someone help me please ?
Thanks !
(P.S : I'm using Python 3.6 on Anaconda)
EDIT : Aaron solved, in this specific case, the problem I had. That being said I'd still be interested to know how I it could be done if one notices something in the source code and wants to modify it.
I have installed virtualenv and the virtualenvwrapper in Powershell for Python 2.7. I am following the tutorial from Tyler Butler listed here
http://www.tylerbutler.com/2012/05/how-to-install-python-pip-and-virtualenv-on-windows-with-powershell/
This is a great tutorial and I have been successful on everything so far after some Google-Fu here and there.
When I attempt to make a new virtualenv by entering:
New-VirtualEnvironment engineer
I get an error which reads:
virtualenvwrapper could not find virtualenv in your PATH
I have noticed there a number of SO posts pertaining to this question and some of them have been informative but nothing has illustrated to me the mistake I am making.
I am a beginner programmer, as you can probably tell.
Steps I have taken:
added virtualenv to my directory path...and that did nothing.
I have looked around in the virtualenv and virtualenvwrapper files for some sort of helpful .txt file but again while informative I could not seem to glean an actual solution.
Any input is appreciated, Please and Thank You for stopping by to read this!
I've just moved from my laptop to a new PC and am having trouble setting up my environment - I can't seem to find the docs I used initially.
That being said, this might help (though it's not the way my initial setup worked):
Set up a powershell profile
Add "Import-Module virtualenvwrapper" (without quotation marks) to your profile.
As to setting up a profile, there should be documentation floating around. Mine is set up with "posh-git", since I was looking to add git functionality. If you're interested in that, read this: http://jdhnet.wordpress.com/2014/01/10/a-better-git-commandline-by-integrating-git-with-powershell/
However, I'm not sure this addresses your particular error, so do reply if it doesn't work - it might be that you're not adding to the path correctly.
Hope that helps! I'll probably edit it later once I have time to read up on .profiles, but this should suffice! Have fun.
First, props to whoever did node.js. I've been using it for less than a day and I'm already thinking about using it for stuff I use Python for now.
In fact, whoever did node.js should think about using it for stuff they use Python for now. There is apparently a tool called node-waf that is in Python and is necessary for npm to work and npm of course is necessary for anything else useful.
I think that my original install went bad because node-waf (which is in /mnt/michael/bin/node-waf) couldn't find Scripting.py (which is in /mnt/michael/node/tools/wafadmin/; it was looking in non-existent /mnt/michael/node/tools/../lib/node/wafadmin/). So I hacked node-waf to point to the right director and kept going and found a much more serious problem.
Turns out node-waf isn't written in "Python", but in Python2.6, which is a perfectly good language, it's the language I use myself, but it isn't the language that the default on the system I use. The system is CentOS, which requires Python2.4 be the Python that the command "python" invokes. Yes, that's foolish on the part of the CentOS people but less foolish than the same mistake on the part of the node-waf people, since they are necessarily subject to the rules of the OS.
So, please tell me there's some switch I haven't found yet that say "Use Python2.6". Also, any hints about a proper install would be appreciated.
To solve the path problem, I backed up and re-installed Node. To solve the version problem, at the suggestion of some bright soul on the #nodejs channel, I created a symbolic link at ~/bin/python that pointed to the right version (that solved a lot of my own problems too, starting up the wrong version from the command line...) An obvious hack, but when you're frustrated, you overlook the obvious.
The Story
After cleaning up my Dreamhost shared server's home folder from all the cruft accumulated over time, I decided to start afresh and compile/reinstall Python.
All tutorials and snippets I found seemed overly simplistic, assuming (or ignoring) a bunch of dependencies needed by Python to compile all modules correctly. So, starting from http://andrew.io/weblog/2010/02/installing-python-2-6-virtualenv-and-virtualenvwrapper-on-dreamhost/ (so far the best guide I found), I decided to write a set-and-forget Bash script to automate this painful process, including along the way a bunch of other things I am planning to use.
The Script
I am hosting the script on http://bitbucket.org/tmslnz/python-dreamhost-batch/src/
The TODOs
So far it runs fine, and does all it needs to do in about 900 seconds, giving me at the end of the process a fully functional Python / Mercurial / etc... setup without even needing to log out and back in.
I though this might be of use for others too, but there are a few things that I think it's missing and I am not quite sure how to go for it, what's the best way to do it, or if this just doesn't make any sense at all.
Check for errors and break
Check for minor version bumps of the packages and give warnings
Check for known dependencies
Use arguments to install only some of the packages instead of commenting out lines
Organise the code in a manner that's easy to update
Optionally make the installers and compiling silent, with error logging to file
failproof .bashrc modification to prevent breaking ssh logins and having to log back via FTP to fix it
EDIT: The implied question is: can anyone, more bashful than me, offer general advice on the worthiness of the above points or highlight any problems they see with this approach? (see my answer to Ry4an's comment below)
The Gist
I am no UNIX or Bash or compiler expert, and this has been built iteratively, by trial and error. It is somehow going towards apt-get (well, 1% of it...), but since Dreamhost and others obviously cannot give root access on shared servers, this looks to me like a potentially very useful workaround; particularly so with some community work involved.
One way to streamline this would be to make it work with one of: capistrano/fabric, puppet/chef, jhbuild, or buildout+minitage (and a lot of cmmi tasks). There are some opportunities for factoring in common code, especially with something more high-level than bash. You will run into bootstrapping issues, however, so maybe leave good enough alone.
If you want to look into userland package managers, there is autopackage (bootstraps well), nix (quickstart), and stow (simple but helps with isolation).
Honestly, I would just build packages with a name prefix for all of the pieces and have them install under /opt so that they're out of the way. That way it only takes the download time and a bit of install time to do.