Agent


You can send your logs to Logentries in a number of different ways. You can use syslog forwarding, the Logentries Agent or one of our platform or language plugins.

What is the Logentries Linux Agent?

The Logentries Agent is a simple program that automates the collection and forwarding of log events and provides a range of useful features. It is open source so you can see exactly how it works and modify it to suit your needs if required.

What Are the Benefits of the Logentries Linux Agent?

  • Allows you to send your logs to Logentries with a quick installation and registration process
  • Configurable through a command line interface
  • Add Hosts and Log files to follow or unfollow in a simple process
  • Allows for filtering / scrubbing of log data before it leaves your servers
  • Quickly view a list of all logs being followed on a Host computer using sudo le whoami command

Setting Up / Installation of Logentries Linux Agent

Requirements

The Linux Agent requires Python 2.6 or higher. We recommend Python 2.7 or higher to ensure reliability. You may also need to install python-setuptools. On Debian-based distributions, this can be accomplished by running:
sudo apt-get install python-setuptools
On RedHat-based distributions, run:
sudo yum install python-setuptools

Installation

Install The Agent

Copy and paste the following snippet into your terminal to install the agent:

wget https://raw.github.com/logentries/le/master/install/linux/logentries_install.sh && sudo bash logentries_install.sh

For instructions on manual installation, choose from the list of supported distros below:

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
echo 'deb http://rep.logentries.com/ wheezy main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install python-setproctitle logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ trusty main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ saucy main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ raring main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ quantal main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install python-setproctitle logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ precise main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install python-setproctitle logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ oneiric main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install python-setproctitle logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ natty main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install python-setproctitle logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ maverick main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ lucid main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -sH
echo 'deb http://rep.logentries.com/ karmic main' > /etc/apt/sources.list.d/logentries.list
gpg --keyserver pgp.mit.edu --recv-keys C43C79AD && gpg -a --export C43C79AD | apt-key add -
apt-get update
apt-get install logentries
le register
apt-get install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
tee /etc/yum.repos.d/logentries.repo <<EOF
[logentries]
name=Logentries repo
enabled=1
metadata_expire=1d
baseurl=http://rep.logentries.com/rh/\$basearch
gpgkey=http://rep.logentries.com/RPM-GPG-KEY-logentries
EOF
yum update
yum install logentries
le register
yum install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
yum-config-manager --add-repo http://rep.logentries.com/helpers/fedora/logentries.repo
yum install logentries
le register
yum install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
yum-config-manager --add-repo http://rep.logentries.com/helpers/fedora20/logentries.repo
yum install logentries
le register
yum install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
tee /etc/yum.repos.d/logentries.repo <<EOF
[logentries]
name=Logentries repo
enabled=1
metadata_expire=1d
gpgcheck=0
baseurl=http://rep.logentries.com/centos5/\$basearch
EOF
yum update
yum install python-simplejson logentries
le register
yum install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
tee /etc/yum.repos.d/logentries.repo <<EOF
[logentries]
name=Logentries repo
enabled=1
metadata_expire=1d
baseurl=http://rep.logentries.com/centos6/\$basearch
gpgkey=http://rep.logentries.com/RPM-GPG-KEY-logentries
EOF
yum update
yum install logentries
le register
yum install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

su -
tee /etc/yum.repos.d/logentries.repo <<EOF
[logentries]
name=Logentries repo
enabled=1
metadata_expire=1d
gpgcheck=0
baseurl=http://rep.logentries.com/rh/\$basearch
gpgkey=http://rep.logentries.com/RPM-GPG-KEY-logentries
EOF
yum update
yum install logentries
le register
yum install logentries-daemon

Run the following commands one by one in your terminal:

You will need to provide your Logentries credentials to link the agent to your account.

sudo -s
tee /etc/yum.repos.d/logentries.repo <<EOF
[logentries]
name=Logentries repo
enabled=1
metadata_expire=1d
baseurl=http://rep.logentries.com/amazon\$releasever/\$basearch
gpgkey=http://rep.logentries.com/RPM-GPG-KEY-logentries
EOF
yum update
yum install logentries
le register
yum install logentries-daemon

First, you have to download Logentries Overlay.

Create a folder to store the package files:

X=/usr/portage/local/app-admin/le
mkdir -p $X/profiles
wget -r -np -nd  -A "*.ebuild","Manifest" -P $X \
http://rep.logentries.com/gentoo/portage/app-admin/le/
echo 'le' > $X/profiles/repo_name

You can also simply add the 'le' folder to an existing overlay, and install the package immediately from there.

Then, you have to add the overlay to Gentoo. Open /etc/make.conf and append the following:

PORTDIR_OVERLAY="${PORTDIR_OVERLAY} /usr/portage/local/app-admin/le"

And install the Logentries package:

emerge app-admin/le

This will install le and all of its components.

Running as a service

To start the service manually, enter as root:

/etc/init.d/logentries start

When you change the configuration, enter as root:

/etc/init.d/logentries restart

Upgrade

Download any newer version of the package:

cd /usr/portage/local/app-admin/le/
wget -r -np -nd -A "*.ebuild","Manifest" -P /usr/portage/local/app-admin/le/ \
http://rep.logentries.com/gentoo/portage/app-admin/le/

And update it:

emerge app-admin/le
Run the Logentries Agent from source

Run the following commands in a terminal to start the agent:

git clone git://github.com/logentries/le.git
cd le
./le register
./le monitor &

Uninstall the Agent

To remove the agent run the following commands:
sudo service logentries stop
sudo apt-get remove logentries-daemon
sudo le clean
sudo apt-get remove logentries

Uninstall the Agent

To remove the agent run the following commands:
sudo service logentries stop
sudo apt-get remove logentries-daemon
sudo le clean
sudo apt-get remove logentries

Follow Your Logs

You can choose which logs to follow by running the le follow command. For example, if you want to follow an Apache Access log file, run:
sudo le follow /var/log/apache2/access.log --name Access
The above can be repeated for as many logs as desired. Note that the --name flag is optional and is used to name the log in your Logentries account. Once you’ve followed your desired logs, restart the agent by running:
sudo service logentries restart

Wildcards

Logs subject to rotation or rollover are often renamed using a sequential number or the current timestamp. In these cases, you can use wildcards to ensure the agent continues forwarding your logs after the file has been renamed.

For example, the following patterns can be used with the follow command to gather logs from the specified directories:
sudo le follow "/var/log/mysystem/mylog-*.log"
Please note that when using wildcards, the path or filename needs to be wrapped in quotation marks (“).

Follow log files through your configuration file

In addition to following logs using the follow command, you can configure log files to be followed locally via the agent’s configuration file. Locally configured logs use token-based inputs, which allow you to collect logs from multiple sources/hosts and forward them to the same destination log in Logentries.

This feature can be particularly useful in an auto scaling environment. You can reuse the same configuration file with multiple agents without creating new Hosts or logs in Logentries. The agent’s configuration file is located at /etc/le/config. Each log you wish to follow will need a separate section in the configuration of the form:
[name]
path = /path/to/log/file
token = MY_TOKEN
Where:
name is an identifier for the application that will be prepended to the log events sent to Logentries
path is an absolute path to the file you wish to follow
token is the token for destination log in your Logentries account


Using local configuration only

In an auto scaling environment you may not want to create a Host each time you install the agent. This can be accomplished by disabling pull-server-side-config, which will stop the agent from communicating with the Logentries API. Add the following line in the [Main] section of the configuration to disable pulling the server-side configuration:
pull-server-side-config=False

Unfollow Log Files

To stop following a log file, use the le rm command as shown below:

1. Get a list of logs the agent is following on your host by running:
sudo le ls hosts/<HOST_MACHINE_NAME>/
For example, you would run the following for a host named “linuxbox”:
sudo le ls hosts/linuxbox/
If you’re not sure what your machine’s hostname is, you can find it by copying and pasting the following into your terminal:
echo $HOSTNAME
2. Instruct the agent to unfollow the log:

Run the le rm command using the log you obtained in Step 1:
sudo le rm /hosts/linuxbox/myLog.log
You will then receive confirmation that the log has been removed:
Log myLog.log removed

Logging to DataHub

Existing installation

To configure an existing install of the agent to send logs to the DataHub, run the command below and replace <DATAHUB_IP> with the IP of your DataHub host:
sudo le reinit --datahub=<DATAHUB_IP>:10000 --suppress-ssl
Next, restart the agent by running:
sudo service logentries restart
The agent will then send logs directly to the DataHub.


New installation

New installations of the agent can be configured to log directly to the DataHub without creating a Host in your Logentries account. For Debian-based Linux distributions, this can be accomplished by following the steps below:
  1. Download the latest Debian file onto your drive, e.g. logentries_1.4.9_all.deb.
  2. Run the following as separate commands:
    sudo dpkg -i logentries_1.4.9_all.deb
    (or the current package name)
    sudo le reinit --pull-server-side-config=False --datahub=<YOUR_DATAHUB_IP>:10000 --suppress-ssl
  3. Add the following to your /etc/le/config file for each log you wish to forward to the DataHub. Make sure to change the values where necessary.
    [YourLogName_OR_YourAppName]
    path = /path/to/log/file
  4. Run:
    sudo apt-get install logentries-daemon
Your agent will now begin to send logs to the DataHub.

Filtering Sensitive Data

We do not recommend storing user sensitive data in log events. However, there may be some cases where this is unavoidable. If you need to obfuscate sensitive data (e.g. credit card information or customer email addresses) in your logs you can do so by following the steps below:

  1. Create a directory called /etc/le/le_filters. This is where you will store a file called filters.py which will filter your log events.
  2. Create an empty file entitled __init__.py in etc/le/le_filters to set up a Python module.
  3. Create a filters.py file in the same directory. This is the file you’ll add your filters dictionary to.
  4. Edit filters.py to include your filtering function and dictionary. Be sure to list your definition of the filtering function first, e.g.:
    #Define your function, in this example our code replaces the word "hello" with "goodbye"

    def filter_shorten_event( events):
    return events.replace("hello", "goodbye")

    #Name your filter functions to be used on a specific log

    filters={
    "mylog.log":filter_shorten_event,
    }
  5. Add the file path to your filters file to your agent’s config file, which can be found at /etc/le/config. Your config file may appear as below:
               [Main]
    user-key = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    agent-key = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    filters = /etc/le/le_filters/
  6. Restart the Logentries agent by running:
    sudo service logentries restart
    NOTE: Each time you change your filters.py file or /etc/le/config file, you must restart the Logentries agent in order for the changes to take effect.
Filtering functions receive a single string containing log entries terminated with a new line. Function can modify lines in any way and return them back for sending to Logentries servers. Do not forget to keep new line termination.

You can also use regular expressions to obfuscate information in your logs. For example, if you need to filter credit card information in your logs, your filters.py file might appear as below:
import re

def filter_credit_card( events):
return CREDIT_CARD.sub( CC_REPLACEMENT, events)

# Credit card number matcher
CREDIT_CARD = re.compile( r'\d{4}-\d{4}-\d{4}-\d{4}')
# Credit card number replacement
CC_REPLACEMENT = 'xxxx-xxxx-xxxx-xxxx' # '-'.join( ['x'*4]*4) if you prefer

Filtering File Names

If you want to explicitly restrict which files the agent can follow, create the filters module and define the filter_filenames function as described in the previous section. The filter_filenames function accepts the full path to a file which is about to be followed. The function returns True if the file name is acceptable and False otherwise. The agent will ignore files that do not pass this test. The following example defines a filter that allows the agent to follow only files with an extension of “.log”:
def filter_filenames( filename):
return filename.endswith( '.log')
You can also define a filter that will restrict the agent to only follow logs within a given directory, for example:
def filter_filenames( filename):
return filename.startswith( '/var/log/')
Please note that the examples above examples do not account for symbolic links.

List of Commands

Commands can be issued to the agent using the following syntax:
sudo le COMMAND [ARGS]
For best results, always ensure you issue commands to the Agent as root/administrator.

Where COMMAND is one of:
  • init Write local configuration file
  • reinit As init but does not reset undefined parameters
  • register Register this host – Registers this host with Logentries and 
  • creates a new host/group in your Logentries account
    • --name= Name of the host
    • --hostname= Hostname of the host
    • --account-key= Your Logentries account key. When installing the agent manually, you can add this option to bypass the prompt for email and password.
  • whoami Displays settings and information for this host, including logs currently followed.
  • monitor Monitor this host’s activities
  • follow <filename> Follow the given log
    • --name= Name of the log that will appear in Logentries
    • --type= Specify log type
  • followed <filename> Check if the file is followed
  • clean Removes configuration file
  • ls List internal filesystem and settings: <path>
  • rm Remove/unfollow log: <path>
  • pull Pull log file: <path> <when> <filter> <limit>
Where ARGS are:
  • --help Show usage help and exit
  • --version Display version number and exit
  • --account-key= Set account key and exit
  • --host-key= Set local host key and exit, generate key if key is empty
  • --no-timestamps No timestamps in agent reportings
  • --force Force given operation
  • --datahub Send logs to the specified data hub address the format is address:port with port being optional
  • --suppress-ssl Do not use SSL with API server
  • --yes Always respond yes

System Metrics

The agent can collect system metrics from CPU, memory, network, disk, and processes. This can be enabled in your agent’s configuration file, which is located at /etc/le/config. Follow the steps below to enable system metrics collection:
  1. Create a new log under the relevant Host in your Logentries account and retrieve the token. This is the destination log that your system metrics will be forwarded to.
  2. Open your agent’s configuration file, which is located at /etc/le/config/. Add metrics-token = YOUR_LOG_TOKEN under the [Main] section of the configuration file
  3. Restart the agent by running
    sudo service logentries restart
A sample agent configuration file appears below:
[Main]
user-key = YOUR_ACCOUNT_KEY
agent-key = YOUR_AGENT_KEY
metrics-interval = 5s
metrics-token = YOUR_LOG_TOKEN
metrics-cpu = system
metrics-vcpu = core
metrics-mem = system
metrics-swap = system
metrics-net = sum eth0
metrics-disk = sum sda4 sda5
metrics-space = /

Fields

The following fields can be used to determine which system metrics are gathered by the agent:
  • metrics-cpu Collects CPU metrics. Allowed values are system which will normalize usage of all CPUs to 100%, or core which will normalize usage to single CPU.
  • metrics-vcpu Collects metrics for each individual CPU. The only allowed value is core, which will normalize usage to a single CPU.
  • metrics-mem Collects memory metrics. The only allowed value is system
  • metrics-swap Collects swap area metrics. The only allowed value is system
  • metrics-net Collects metrics for specified network interfaces. Allowed values are interface IDs (e.g. eth0). Special interfaces are:
    • all which instructs the agent to follow all interfaces (including lo)
    • select which will follow selected interfaces such as eth and wlan
    • sum which aggregates usages of all interfaces in the system
  • metrics-disk Collects disk IO metrics. Allowed values are device IDs (e.g. sda4) and all, which instructs the agent to collect metrics for all devices.
  • metrics-space Collects disk space metrics. Allowed values are device IDs (e.g. sda4)
  • metrics-process Collects metrics for a specific process. This parameter should be specified in a separate section as shown below:
    [cassandra]
    metrics-process = org.apache.cassandra.service.CassandraDaemon

Troubleshooting

Should you experience any difficulties with the agent, please follow these steps to troubleshoot.

Cannot register the machine

This is likely caused by firewall issues. The agent requires to open a connection tologentries.comandapi.logentries.comon a secure port 443 or 80. Run the agent with the--debugargument to print the sequence of executed commends.
le --debug register

I don’t see any statistics displayed

Likely, the daemon is not running or there is a firewall on the way. Note that statistic collection works on Linux systems only. To check that the process is running, run the following command:
ps aux|grep logentries-daemon
The running agent should be displayed. If not, the agent is not running and can be started running this command:
service logentries start
If the agent is running, make sure it can connect to Logentries servers. Firstly, stop the background daemon as root:
service logentries stop
Then, run the monitor command on the command line with debugging enabled. This is similar to daemon mode with a console attached:
le --debug monitor
Any issues will be displayed on console.

Logs are not sent to Logentries

Firstly, make sure that the daemon is running and you see statistics collected. If you still don’t see any new log entries, make sure that the log file name has been entered correctly. To do that, list the log properties:
le ls hosts/HOSTNAME/LOGNAME/
ChangeHOSTNAMEandLOGNAMEto the host name and log name respectively. The output will look like this:
name = Auth
filename = /var/log/auth.log
key = cbe2d8ff-96d1-44e5-bba2-71fcdb162825
type = filename
follow = true
Check that the file with the name given really exists:
ls 'filename'
If you get an error message, the file name is incorrect. Otherwise check that new log entries appears in the log file via:
less 'filename'
and pressFto display new log entries. If no new entries appear then there are no new entries to be sent to Logentries. Otherwise continue debugging the agent. Stop the daemon and run it on a command line with debugging enabled:
service logentries stop
le --debug monitor
If you see any error messages, resolve them. If all appears to be ok, check that log entries are captured correctly by enabling dump on console. Break the monitoring withCtrl+Cand run again:
le --debug --debug-events monitor
If no log entries appears on console then the agent cannot retrieve new entries from the file. Otherwise all is OK.
Back to Top