How to Write syslog to a Remote Server Using Python

Introduction:

This is a follow-up post for writing syslog using python. In my previous article, we discussed on how to write syslog to a local host using python. We have also walked through what a syslog is and its internal workflow. If you have landed here directly and you want a quick refresher on syslog, please read this article.

I assume that you have a basic idea of syslog now and without further ado, let’s quickly jump into writing syslog to a remote server from a python application. However, those are not the only things we are going to talk about.

We will also be talking about 2 additional use cases here although they are not very different from each other.

1. Forward syslog from a docker container to a remote server at application level.
2. Forward syslog from a docker container at daemon/container level

1. Writing syslog to a remote server from python application:

Let’s say you have the following python module.

main.py

from logging import getLogger, Formatter
import logging.handlers


# Make the desired format string for Syslog
LOG_FORMAT = f"%(levelname)s:%(filename)s:%(lineno)d - %(asctime)s - %(message)s"


# Get the root logger
logger = getLogger()

syslogHandler = logging.handlers.SysLogHandler(address=("<your remote server>",
514))

logger.setLevel(logging.INFO)

# Set the format string for the handler
syslogHandler.setFormatter(Formatter(LOG_FORMAT))

logger.addHandler(syslogHandler)
logger.info("Configuring logger")
logger.info("Logging configured!")

This code will forward logs to a remote server at the application level even if you containerise this application.

If you’re interested in forwarding logs from a python application at the container level, please read further.

2. Writing syslog to a remove server using Docker:

Now that we have the application code ready, let’s see what Docker provides to support syslog.

We will also discuss how to forward logs at container level.i.e. This is required if you have a VM running multiple docker containers and you don’t want to go through the hassle of configuring syslog for each container.Despite the log configuration at the application level, configuring syslog at container level will forward the logs to the specified remote server.

Logging Drivers in Docker:

Docker provides multiple logging mechanisms called logging drivers. By default, the docker daemon has a log-driver unless we override with a different one. The default log driver is the json logging driver.

https://docs.docker.com/config/containers/logging/configure/

For our use-case, the logging driver will be syslog . A list of supported logging drivers is available in the below link.

https://docs.docker.com/config/containers/logging/configure/#supported-logging-drivers

You could either configure the logging drivers at the container level or at the daemon level. Let’s discuss both.

i. Configuring logging drivers at daemon level:

The logging drivers can be set at the daemon level using the docker config file located at /etc/docker/daemon.json on ubuntu systems.

Note: By default, this file may not be available. Please feel free create one in the aforementioned location.

{
	"log-driver":  "syslog",
	"log-opts":  {
	"syslog-address":  "tcp://<your remote server>:514"
	}
}

This ensures all the container logs to be forwarded to the specified remote server. An advantage of configuring drivers at daemon level is, you don’t have to specify the logging driver for every single container even if you have multiple containers.

You could always override the logging driver for a specific container in the docker run command using — log-driver flag.

ii. Configuring logging drivers at container level:

The docker run command includes a -log-driver and — log-optflag where you may specify the log driver type and the log address.

Build the docker container:

docker build -t test-syslog .

Run the docker container:

docker run --log-driver syslog --log-opt syslog-address=<your remote server>:514 test-syslog

A server listens to syslog in port number 514.The log driver type is syslog and the log options has the syslog remote server address and port number.

Let’s take a look at the Docker file.

FROM python:3.10-bullseye

WORKDIR /app

COPY ./ ./


EXPOSE 514/tcp

EXPOSE 514/udp

RUN echo  "Executing script"


ENTRYPOINT ["python3", "main.py"]

Summary:

• Forwarding container or application logs to a remote server makes operations and maintenance easier. It also helps for analysis and monitoring.
• Logs can be forwarded from either the application or from the container.
• Docker supports variety of logging drivers which includes syslog
• Logging drivers can be configured at daemon level and container level.