How to tail (follow) Linux Logs

/in , /by

If you’re wondering how to follow Linux logs for a process or systemd unit (service), here are the commands you want:

Traditional, File-based logs

For a traditional service (a long-running process that logs to one or more files), you can use traditional Linux tools to check the logs. A common command might be:

tail -f /var/log/$YOURLOGFILE

Replace $YOURLOGFILE with the actual name of the logfile — the actual log path may be different, it just depends on what your process/service is configured to use.

The ‘tail’ command will start at the end of the logfile, and the ‘-f’ option will follow the log as new lines are written to it. You’re basically getting a ‘live’ view of the log as it grows.

For a systemd service (systemd unit)

Systemd uses journald (the systemd log journal) to create binary logs (non-plaintext logfiles, i.e. the text would look garbled if you looked at it with traditional Unix/Linux tools like less, tail, cat, etc.).

To view logs through journald, you’ll want these commands:

journalctl $YOURSERVICE

Just calling journalctl with a service (systemd unit) name will get you all logs for that service, starting from the beginning. You can page through with the space bar and quit with the ‘q’ key. Prepare to do a lot of scrolling. A more practical command is:

journalctl -fu $YOURSERVICE

This tells journalctl to do the same thing as “tail -f” — it starts at the end of the logs (the most current ones) and follows new logs ‘live’ as they stream in.

I made a video on this (it also contains a bit more context and an extra trick or two); enjoy!

How to Install Ansible on Ubuntu Linux

/in , /by

The official Ansible installation instructions are WRONG, and will result in a bunch of errors and wasted time troubleshooting. Here’s how to install the newest version of Ansible on Ubuntu or Debian (or any other Linux distro, provided you swap out aptitude for your own package management commands).

Note:  This installs the NEWEST ‘stable’ version of Ansible, even if Ubuntu’s package repositories are outdated (which they usually are):

sudo -i
apt-get install pip
pip install --upgrade pip
pip install ansible jinja2 pyaml

Troubleshooting Failed Installs Because You Naiively Followed the Official Instructions

Here are the official installation docs, via https://docs.ansible.com/ansible/latest/intro_installation.html#latest-releases-via-pip

# Not really all you need to do
root@jenkins2:~# pip install ansible

First Error: Ansible doesn’t automatically pull in the required jinja2 module/package

Here’s what this looks like, after you run ‘pip install ansible’ as above:

jenkins@jenkins2:~$ ansible --version
Traceback (most recent call last):
 File "/usr/local/bin/ansible", line 40, in <module>
 import ansible.constants as C
 File "/usr/local/lib/python2.7/dist-packages/ansible/constants.py", line 12, in <module>
 from jinja2 import Template
ImportError: No module named jinja2

 

You get this error because ansible requires the jinja2 module, which isn’t marked as a required package in pip for some reason.

 

Solution: Install the missing jinja2 module via the jinja2 package in Pip

sudo pip install jinja2

 

Second Error: Pip doesn’t automatically pull in a yaml package for Ansible

This is the case if you’re getting an error like the following one:

jenkins@jenkins2:~$ ansible --version
Traceback (most recent call last):
 File "/usr/local/bin/ansible", line 40, in <module>
 import ansible.constants as C
 File "/usr/local/lib/python2.7/dist-packages/ansible/constants.py", line 18, in <module>
 from ansible.config.manager import ConfigManager, ensure_type, get_ini_config_value
 File "/usr/local/lib/python2.7/dist-packages/ansible/config/manager.py", line 11, in <module>
 import yaml

 

Solution: Install the yaml module via the pyaml package in Pip

sudo pip install pyaml

This is a stupidly named package in Pip — it’s easy to get the name wrong, as below. The solution is to just install the ‘pyaml’ package instead.

# ERROR: Install pyaml (it's not called yaml)
root@jenkins2:~# pip install yaml
Collecting yaml
 Could not find a version that satisfies the requirement yaml (from versions: )
No matching distribution found for yaml

 

So if you followed the instructions at the beginning of this post, OR you had to troubleshoot and start in the middle of this post somewhere, you should now have a functioning ansible install. If running ‘ansible –version’ doesn’t print out a version number, it should at least point you to the next error you need to troubleshoot.

Here’s an example from a Jenkins build server:

jenkins@jenkins-server:~$ ansible --version
ansible 2.4.2.0
 config file = /etc/ansible/ansible.cfg
 configured module search path = [u'/var/lib/jenkins/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
 ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
 executable location = /usr/local/bin/ansible
 python version = 2.7.12 (default, Nov 20 2017, 18:23:56) [GCC 5.4.0 20160609]

 

I hope that helps to guide you through the frustrating path of bad package naming, outdated repositories, and broken tooling that is (seemingly always) the current state of affairs in Ansible-land!

By the way, if you want to learn more about Linux, Programming, and other tech stuff, I’ve got a bunch of free Linux video tutorials up on my YouTube Channel: https://www.youtube.com/c/tutorialinux

Check it out!

Amazon AWS Basics Tutorial: Setting Up a Load-Balanced, Auto-Scaling Webserver

/in , /by

This step-by-step tutorial will show you how to build load-balanced, highly available, self-healing infrastructure on the Amazon Web Services (AWS) Cloud.

If you’ve been wondering how to get started with “DevOps,” “Cloud” System Administration, and public cloud providers in general, this is the series for you.

Amazon’s Cloud Services are vast — there are an enormous amount of things you can use:

  • services that replicate traditional Virtual Machines in a datacenter (EC2, RDS, Elasticache)
  • services that give you insane amounts of availability and durability, along with an innovative API (S3 for storage)
  • replacements for other solutions in brand new industries (ECS)
  • plenty of unique services that no one else is offering

 

If you really want to learn this stuff, you want to start with simple, practical examples that slowly walk you through the huge amount of new concepts that you’ll need to learn. You need something practical that you can use in real life, so that navigating Amazon’s immensely complex graphical user interface becomes second-nature.

Once you’ve got a grasp on the basic concepts and the most important services, we can dive into building more realistic, more complex infrastructure that uses specific AWS features to get the job done.

We’ll also explore the AWS API, which allows you to use a command-line client or bindings for your favorite programming language to work with infrastructure. This is the preferred way that professionals use to interact with AWS.

 

Video #1: Intro and EC2 Instance Creation

In the first video, you’ll learn the basics of working in the Amazon AWS Console (their graphical user interface) and set up an EC2 instance with a webserver on it. You’ll create your first security group (AWS network firewall rules), set up SSH keys, and connect to your ‘development’ instance for the first time.

Once you’re ready for launch, you’ll learn how to create a new Amazon Machine Image (AMI) from your EC2 instance, which we can use to spin up clones of that instance later on.

 

Video #2: Auto-Scaling Groups and Launch Configurations

In the second video, I’ll show you how to set up an Auto-Scaling Group (ASG) which uses a Launch Configuration to spin up new EC2 instances (clones of your development instance). We’ll talk a bit about scaling and how you’d set it up in a real project.

 

Video #3: Tying it together with an Elastic Load Balancer

In the third video, you’ll be setting up an Amazon Elastic Load Balancer (ELB) to balance traffic across the instances in your Auto-Scaling Group. I’ll show you how to add health checks and hook up the Application Load Balancer to your ASG, so it automatically balances across healthy instances in your ASG, even as they’re added and removed.

 

There’s more coming, so stay tuned!

Cron Jobs

/1 Comment/in , /by

Cron jobs allow you to repeatedly run commands. They can be run both in a timed manner, but also on bootup. While cron may seem a bit confusing in the beginning, it’s actually very simple to use once you know how it works.

Read more

man man – What does the 1 in ls(1) mean?

/in , /by

man pages – many people would have loved to have known earlier about them. This article could be summed up as “just read man-pages(7)”. But what does that mean and why should you care?

Read more

Git: Setting up a low-requirements, personal Git repository

/in , , /by

In this tutorial you will learn how to  how to quickly set up a simple, low-requirements, no-database, no-webserver Git repository. All you need is (unprivileged) SSH access and the git program itself.

Read more

The System Administration Course on Udemy is now $12 (from $45)!

/5 Comments/in , /by

It’s been several months since I released my project-based system administration course on Udemy (https://www.udemy.com/hands-on-linux-self-hosted-wordpress-for-linux-beginners/?couponCode=SYSADMINW12).

To celebrate a big move back to the U.S. (and a new job), I just lowered the price from $45 to $25. As always, the tutorialinux sysadmin horde gets a better deal: $12 for all 8 hours of the video course.

For those that don’t know about the course, it’s titled “Hands-on Linux: Self-Hosted WordPress for Linux Beginners” and will teach you the basics of Linux system administration using a real-life project. You’ll set up a WordPress website, and the infrastructure to run further websites (as many as you want).

More info can be found in my original post about the course.

I also show you the sysadmin skills that you generally don’t see in tutorials or YouTube videos: automation, security hardening, backups and restores, next steps and add-on project ideas.

There’s a 30-day money back guarantee on Udemy, so there’s no risk if you find that the course isn’t for you.

Have fun!

https://www.udemy.com/hands-on-linux-self-hosted-wordpress-for-linux-beginners/?couponCode=SYSADMINW12

Shell Aliases

/in , /by

When working on Linux and Unix systems, you’ll often find yourself using long shell commands that you repeat several times per day (or per session). Things like

  • checking on the current resource usage of a system,
  • checking if a process has completed yet, or
  • using a complex group of commands, piped together with special options to give you just the output you want.

Generally, you’ll either use CTRL+R to search for these past commands in the shell’s history. Some of you may even copy useful commands to a special file that you can easily reference. You might even just enter them manually each time you need them, since you’re too lazy to be bothered.

There’s a simple solution that covers all of these cases well: Aliases.

Read more

How to convert an SSH2 Public Key into an OpenSSH public key

/in , , /by

When working with people who don’t use a Unix-based operating system, you’ll often come across the SSH2 Public Key format. PuTTY is probably the most famous software using this format and nearly everyone on Windows uses it. To give these windows ssh users access to a Linux system, SFTP server, Git repository or other systems that use the OpenSSH key format, you need to convert an SSH2 public key into the OpenSSH format. This article describes how to do exactly that.

For more Linux and programming tips, tricks, and videos, check out my channel here: https://www.youtube.com/c/tutorialinux — I have a completely free Linux Sysadmin course in this free Linux course playlist.

Okay, onto the openssh key converting goodness!

Read more

How to Install Dwarf Fortress on Ubuntu 16.04

/4 Comments/in /by

Note: If you don’t know what Dwarf Fortress is, you owe it to yourself to learn a bit about it. It’s one of the most interesting and detailed games/simulations/sandboxes ever created.

I just spent a bit of time getting Dwarf Fortress installed on Ubuntu 16.04. I didn’t find much in the way of up-to-date instructions, so I’m writing down what works as of today (June 2016). If you’re wondering how to install Dwarf Fortress on Ubuntu 16.04, here’s what you’ll want to do:

Read more