How to tail (follow) Linux Logs

/in , /by

If you’re wondering how to follow Linux logs for a process or systemd unit (service), here are the commands you want:

Traditional, File-based logs

For a traditional service (a long-running process that logs to one or more files), you can use traditional Linux tools to check the logs. A common command might be:

tail -f /var/log/$YOURLOGFILE

Replace $YOURLOGFILE with the actual name of the logfile — the actual log path may be different, it just depends on what your process/service is configured to use.

The ‘tail’ command will start at the end of the logfile, and the ‘-f’ option will follow the log as new lines are written to it. You’re basically getting a ‘live’ view of the log as it grows.

For a systemd service (systemd unit)

Systemd uses journald (the systemd log journal) to create binary logs (non-plaintext logfiles, i.e. the text would look garbled if you looked at it with traditional Unix/Linux tools like less, tail, cat, etc.).

To view logs through journald, you’ll want these commands:

journalctl $YOURSERVICE

Just calling journalctl with a service (systemd unit) name will get you all logs for that service, starting from the beginning. You can page through with the space bar and quit with the ‘q’ key. Prepare to do a lot of scrolling. A more practical command is:

journalctl -fu $YOURSERVICE

This tells journalctl to do the same thing as “tail -f” — it starts at the end of the logs (the most current ones) and follows new logs ‘live’ as they stream in.

I made a video on this (it also contains a bit more context and an extra trick or two); enjoy!

Amazon AWS Basics Tutorial: Setting Up a Load-Balanced, Auto-Scaling Webserver

/in , /by

This step-by-step tutorial will show you how to build load-balanced, highly available, self-healing infrastructure on the Amazon Web Services (AWS) Cloud.

If you’ve been wondering how to get started with “DevOps,” “Cloud” System Administration, and public cloud providers in general, this is the series for you.

Amazon’s Cloud Services are vast — there are an enormous amount of things you can use:

  • services that replicate traditional Virtual Machines in a datacenter (EC2, RDS, Elasticache)
  • services that give you insane amounts of availability and durability, along with an innovative API (S3 for storage)
  • replacements for other solutions in brand new industries (ECS)
  • plenty of unique services that no one else is offering

 

If you really want to learn this stuff, you want to start with simple, practical examples that slowly walk you through the huge amount of new concepts that you’ll need to learn. You need something practical that you can use in real life, so that navigating Amazon’s immensely complex graphical user interface becomes second-nature.

Once you’ve got a grasp on the basic concepts and the most important services, we can dive into building more realistic, more complex infrastructure that uses specific AWS features to get the job done.

We’ll also explore the AWS API, which allows you to use a command-line client or bindings for your favorite programming language to work with infrastructure. This is the preferred way that professionals use to interact with AWS.

 

Video #1: Intro and EC2 Instance Creation

In the first video, you’ll learn the basics of working in the Amazon AWS Console (their graphical user interface) and set up an EC2 instance with a webserver on it. You’ll create your first security group (AWS network firewall rules), set up SSH keys, and connect to your ‘development’ instance for the first time.

Once you’re ready for launch, you’ll learn how to create a new Amazon Machine Image (AMI) from your EC2 instance, which we can use to spin up clones of that instance later on.

 

Video #2: Auto-Scaling Groups and Launch Configurations

In the second video, I’ll show you how to set up an Auto-Scaling Group (ASG) which uses a Launch Configuration to spin up new EC2 instances (clones of your development instance). We’ll talk a bit about scaling and how you’d set it up in a real project.

 

Video #3: Tying it together with an Elastic Load Balancer

In the third video, you’ll be setting up an Amazon Elastic Load Balancer (ELB) to balance traffic across the instances in your Auto-Scaling Group. I’ll show you how to add health checks and hook up the Application Load Balancer to your ASG, so it automatically balances across healthy instances in your ASG, even as they’re added and removed.

 

There’s more coming, so stay tuned!

Sysadmin Links: Windows Package Management, SSH Wizardry, and Strategy Games!

/in , /by

Another episode of the “Sysadmin Timewasters” series just went up on YouTube. In this episode, we’re looking at several interesting projects:

0:01 Keep your eyes healthy! https://tutorialinux.com/want-to-keep-your-eyes-healthy-use-redshift/

3:01 How to choose a programming language: https://tutorialinux.com/which-programming-language-should-i-choose/

 

Read more

Linux Command-Line Basics: Two Videos

/in , /by

If you want to be a competent Linux or Unix Administrator, Developer, or IT Person, you need to be completely comfortable on the Linux Command Line. For that reason, I’ve approached the “Linux Command-Line Basics” topic from a few different angles so far.

This post should be a good jumping-off point for anyone who wants to dive in — it’s a one-stop shop for all the free videos I’ve made on the subject of Linux command-line basics and basic Bash shell skills over the last three years. Let’s get started!

Read more

Course Launch: Hands-on Linux: Self-Hosted WordPress for Linux Beginners

/2 Comments/in , , /by

It’s taken me several months but I’ve finally done it: this weekend, I’m launching the first tutorialinux course on the Udemy learning platform. The course is called “Hands-on Linux: Self-Hosted WordPress for Linux Beginners.”

https://www.udemy.com/hands-on-linux-self-hosted-wordpress-for-linux-beginners/
It’s a project-based course which teaches the basics of Linux system administration using a practical, real-life project to lead you through the material. In the course, I walk beginning Linux sysadmins through setting up a fully-featured, production-grade WordPress hosting platform on their own server.

Of course, you can run other PHP applications on this platform, too. I chose WordPress because it’s so insanely popular right now, and because I know the platform relatively well after spending a year working as a security consultant doing malware cleanups and security overhauls on compromised WordPress sites.

The course itself follows the project-based learning approach I’ve been talking about recently. Although I think theory is important (and occasionally even fun), people just seem to learn much faster when they work on a practical project that ties together 10 or 20 individual skills and gives them a usable artifact at the end (in this case, a hosting platform).

I supply a slow drip of theory in this course — just enough to keep students making progress on the project while still understanding what’s going on.

 

More than a “Basics” Tutorial

The course is much more than just basic application setup and configuration, though. I’ve made sure to cover “real sysadmin” stuff; the things that sysadmins actually spend their time doing in real life (not just “apt-get install -y somesoftware && nano /etc/configfile”). Topics like:

  • system monitoring
  • performance optimization and caching
  • security hardening
  • creating and restoring website backups (filesystem backups and MySQL backups)
  • HTTP protocol basics

The course features 71 videos right now; about 8 hours of video content. There’s more coming, too: I’ll be continuing to improve and add material to the course as it grows and I get feedback from students.

Plus, you’ll have something to ‘take home with you’ when you finish the course: it’s always cool to have a robust, performant hosting platform at your fingertips, ready to do your bidding, host your friends’ websites, make you millions of dollars, etc.

I’ve marked a bunch of the videos as being ‘free previews,’ so there’s about an hour of viewing to be had for free on the “course curriculum” page.
All the links in this post include a coupon for $7 off the retail price (just over 15%). Have a look at the course curriculum, and check out some of the free preview videos from the course!

Get over there and check it out!

New tutorialinux guide: Getting Started with Linux Containers (LXC)

/in , , , /by

A while back, I did a YouTube series on Linux Containers (LXC). If you are (or want to be) a sysadmin or software developer, you need to know about Linux Containers, and understand how to use them. I’ve just written a ~45-page guide to getting started with this useful skill — check it out here! For those of you that want more details (or a link to the original playlist), read on:

Read more

32c3 Video: Analysis of Red Star, the North Korean OS

/in , , /by

Did you know that Kim Jong Un, the glorious, fearless, and immaculately rotund leader of North Korea, loves Mac OS X (and possibly Madonna)? If not, you’ll want to check out this video about Red Star OS, the operating system which the North Korean government has put together for their citizens.

Read more

The Top 10 WordPress Security Mistakes

/in , /by

Sooner or later, we all deploy a PHP web application (Joomla, WordPress, Magento, etc.). I’m currently doing some security work, and I deal with a huge number of sites that have been hacked. The crazy thing is that 90+ percent of these compromises could have been prevented by a few security precautions. Here’s a list of the most common misconfigurations and security holes (from a System Administrator’s perspective), along with how to fix them.

We also look at some of the most popular goals of the attackers (‘consequences’).

Read more

Where to Find Answers to Linux Questions

/in , /by

When they’re faced with a technical problem — an error message, unexpected behavior, or ‘I have no idea what to do about this problem’ moment — most people resort to Google (or a privacy-aware search engine, like DuckDuckGo or IxQuick). However, often this is not the quickest way to get the problem solved.

Here’s a quick checklist of resources for solving Linux problems:

Read more

Finding out what’s happening on a Linux Machine

/in , /by

Here’s a quick video on finding out what’s going on when you log into (or sit down at) a Linux machine. We cover the following commands:

w (/who): find out who is currently logged into the machine, and where they're connecting from

top: find out which processes are running, how much load they're putting on the CPU, and how much memory they're using.

(sudo) netstat -tulpn: Check for listening network ports and which processes/programs are using them.

Some Alternatives to the Tools I show in this Video:

I really like a program called htop, which shows you a bit more information and presents it in a nicer format. I recommend using it, once you’re comfortable with top.

htop (sudo apt-get install htop -- quit with 'ctrl-c' or 'q' once it's running)

Have fun!