How to Install Ansible on Ubuntu Linux

/in , /by

The official Ansible installation instructions are WRONG, and will result in a bunch of errors and wasted time troubleshooting. Here’s how to install the newest version of Ansible on Ubuntu or Debian (or any other Linux distro, provided you swap out aptitude for your own package management commands).

Note:  This installs the NEWEST ‘stable’ version of Ansible, even if Ubuntu’s package repositories are outdated (which they usually are):

sudo -i
apt-get install pip
pip install --upgrade pip
pip install ansible jinja2 pyaml

Troubleshooting Failed Installs Because You Naiively Followed the Official Instructions

Here are the official installation docs, via https://docs.ansible.com/ansible/latest/intro_installation.html#latest-releases-via-pip

# Not really all you need to do
root@jenkins2:~# pip install ansible

First Error: Ansible doesn’t automatically pull in the required jinja2 module/package

Here’s what this looks like, after you run ‘pip install ansible’ as above:

jenkins@jenkins2:~$ ansible --version
Traceback (most recent call last):
 File "/usr/local/bin/ansible", line 40, in <module>
 import ansible.constants as C
 File "/usr/local/lib/python2.7/dist-packages/ansible/constants.py", line 12, in <module>
 from jinja2 import Template
ImportError: No module named jinja2

 

You get this error because ansible requires the jinja2 module, which isn’t marked as a required package in pip for some reason.

 

Solution: Install the missing jinja2 module via the jinja2 package in Pip

sudo pip install jinja2

 

Second Error: Pip doesn’t automatically pull in a yaml package for Ansible

This is the case if you’re getting an error like the following one:

jenkins@jenkins2:~$ ansible --version
Traceback (most recent call last):
 File "/usr/local/bin/ansible", line 40, in <module>
 import ansible.constants as C
 File "/usr/local/lib/python2.7/dist-packages/ansible/constants.py", line 18, in <module>
 from ansible.config.manager import ConfigManager, ensure_type, get_ini_config_value
 File "/usr/local/lib/python2.7/dist-packages/ansible/config/manager.py", line 11, in <module>
 import yaml

 

Solution: Install the yaml module via the pyaml package in Pip

sudo pip install pyaml

This is a stupidly named package in Pip — it’s easy to get the name wrong, as below. The solution is to just install the ‘pyaml’ package instead.

# ERROR: Install pyaml (it's not called yaml)
root@jenkins2:~# pip install yaml
Collecting yaml
 Could not find a version that satisfies the requirement yaml (from versions: )
No matching distribution found for yaml

 

So if you followed the instructions at the beginning of this post, OR you had to troubleshoot and start in the middle of this post somewhere, you should now have a functioning ansible install. If running ‘ansible –version’ doesn’t print out a version number, it should at least point you to the next error you need to troubleshoot.

Here’s an example from a Jenkins build server:

jenkins@jenkins-server:~$ ansible --version
ansible 2.4.2.0
 config file = /etc/ansible/ansible.cfg
 configured module search path = [u'/var/lib/jenkins/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
 ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
 executable location = /usr/local/bin/ansible
 python version = 2.7.12 (default, Nov 20 2017, 18:23:56) [GCC 5.4.0 20160609]

 

I hope that helps to guide you through the frustrating path of bad package naming, outdated repositories, and broken tooling that is (seemingly always) the current state of affairs in Ansible-land!

By the way, if you want to learn more about Linux, Programming, and other tech stuff, I’ve got a bunch of free Linux video tutorials up on my YouTube Channel: https://www.youtube.com/c/tutorialinux

Check it out!

Thoughts Worth Thinking About

/in , /by

I was just chatting with a friend about software, infrastructure, modern architectures and methodologies, and just sysadmin stuff in general. After getting up for a few minutes to grab a drink, here’s what I found waiting for me in the chat room. I’m sharing it, with his permission:

 

I find it funny. Everyone is like, “my servers are cattle, not pets”, “no snowflakes” and “use Docker,” with the result that the infrastrcture and host systems are the worst form of this. xD

But to be honest, I am realizing that the cattle/pet thing is wrong. I want my server to be a shepherd dog and my processes/services to be cattle. I want my dog to be smart and take care of things and be reliable, exactly because my cattle isn’t.

I want it to do the right thing at the right moment and in the right way. I want to improve my servers every time something goes wrong and not have them be as dumb as the first time it happened.

And the snowflake. No, I don’t want my system to be like every other. In that case I would just make a distro that is like that and just install it. No architecture/design of system/infrastructure required.

It’s like.. just using WordPress instead of starting your own thing. “But that way it’s easier, and you don’t have your snowflake web app.”

Oh and you can google stack traces.

Why program at all?

It’s also a bit like when people thought it was a good idea to use Microsoft Office (or early versions of Frontpage) to make web pages. xD

The WP comparison probably also works with OSs. “Why not just use Ubuntu” is like “why not just use WP?” Big community, widely used language.

 

It’s worth considering. Thinking carefully about fads and hype, before going along with them, is a good thing. And even if a lot of these technologies and approaches have merit, they are not a ‘one size fits all’ answer. If something doesn’t make sense to you, always ask more questions, do more research, and try things out yourself. That’s what makes us better than machines :-).

I was wrong about Docker

/in , , , /by

I recently found an answer that I gave someone to a Facebook chat question in ~2016-2017, and was amazed at how “technically right” I was (i.e. totally wrong for the career question that I was answering).

I just wanted to post this to show that I am often too technologically focused, and that this can often get in the way of getting hired and getting things done the way everyone else is doing them. On the flipside, my heart is in the right place and this approach certainly wouldn’t have HURT the person I was giving advice to.

Here it is!

Yeah, I should make some docker vids! So, LXC is the original Linux-native implementation of containers. There was an implementation before, called OpenVZ, but that relied on a patched Linux kernel and I don’t think it’s used much anymore.

The mainline kernel itself exposes the “ingredients” for containers (user/process namespacing, cgroups, etc.) and LXC was one of the original implementations of ‘containers’ using these native building blocks.

Docker was originally a wrapper around LXC, but has now actually created their own implementation of the container back-end (using the same exposed ‘ingredients’ from the Linux kernel that LXC uses).

LXC (and now LXD) is under-hyped, solid, and a bit more intuitive for learning (more like FreeBSD Jails, less hype/magic than Docker). I think it’s good to learn LXC or FreeBSD jails before jumping into Docker.

All the concepts from LXC are totally applicable to Docker as well.

Docker, as a software product, layers some extra tools on top of this basic ‘containerization’ tech: they have their docker ‘hubs’ — pre-built application containers. They have extra networking stuff on top of what you’d get with LXC. They have something that looks like very basic container linking/service discovery. They have enormous amounts of hype, which can help you get a high-paying job (seriously. It’s worth learning just for that.)

I’m torn, because I’ve used Docker since before it was an open-source product (it used to be a paid thing called DotCloud), and have hated it for almost the entire time. It’s a complex, unstable, and questionably architected piece of software. It’s also incredibly overhyped and misused.

BUT: Docker has some great ideas in it, and is absolutely worth learning for your career. As a technology, it leaves a lot to be desired.

I’d love to hear what you think of Docker after getting comfortable with LXC. LXC will teach you the concepts that underlie all of this containerization stuff, and Docker adds some new features (and new headaches) onto that.

Let me know what you think, if/when you start your Docker journey!

The Hardest (and most fun) Problems to Troubleshoot

/in , /by

I recently wrote a FAQ-style post about System Administration and technology careers in general. One of the best questions I was asked was about what kinds of really interesting troubleshooting problems I’ve had to deal with. Here’s that question, along with my answer:

What’s one of the most interesting things you’ve had to troubleshoot / do while maintaining a system?

I’m leaving out specific examples because they’re a mixture of non-public information and hyperspecific (uninteresting) technical stuff, but I can give some outlines for what generally makes for interesting problems to solve.

The really interesting problems I’ve seen tend to be related to performance, networking, and distributed systems. Usually they require a combination of different knowledge to solve:

  • Systems/OS: What is the operating system doing when everything slows down? What’s causing it to do that?
  • Networking/Distributed Systems: What’s actually happening when these machines communicate? How are they supposed to share and manage state, deal with network partitions, and ensure high availability? What are they *actually* doing when this problem happens?
  • Software Development: Which part of the code is causing this network/OS issue, and which code path leads there? Can I actually look at and modify this code? Is this code written by our developers, or an open-source project? What can I do to confirm the issue and test a fix? Can I contribute a fix back to the upstream project?

How to Record Your Work on the Command Line with the script(1) Command

/in , /by

Most Unix-like operating systems feature a script command. You can find its manual in script(1) (type man 1 script to access it). script records a transcript (“typescript,” not to be confused with the language TypeScript) of your current session in the command line.

The script command can be used as a way to log what you are doing in a shell session. It’s often used during troubleshooting, documentation, PCI compliance audits, security/remediation work, penetration tests, and other situations where it’s useful to record a play-by-play log of what you’re doing on the machine.

 

Practical Demonstration

 

Read more

Today I Learned: Migrating from sqlite to Postgres is easy with Sequel

/1 Comment/in /by

I spent some time migrating an application from sqlite3 to Postgres today, and wanted to write down a few notes for next time. Here they are!

Read more

Today I Learned: ZFS send/receive is Like rsync, but Developed by a Much More Evolved Species

/in /by

If you use ZFS, you already know that it completely knocks the socks off of other filesystems. It prevents and corrects data corruption, gives you incredible flexibility, and basically gives you everything you could ever want from a filesystem. If you use OpenZFS on several systems already, you probably know about the ‘send’ and ‘receive’ commands to do incremental transfer of snapshots between systems.

Here’s a great video on ZFS send and receive, which goes much deeper than most videos into how send and receive are implemented, along with some clever ideas for use cases where send/receive can really save you a lot of time and pain.

Because being able to mimic $50,000 enterprise filesystem replication on your home NAS or your little ‘friends-and-family’ WordPress hosting server is pretty effing cool.

Read more