I’ll write a comprehensive technical blog post about managing log files with logrotate on Ubuntu 24 following your specifications.

Log management is one of those essential sysadmin tasks that can make or break your server’s performance and disk space. Without proper log rotation, your system can quickly run out of storage, crash services, or become impossible to troubleshoot when you actually need those logs. Logrotate is Ubuntu’s built-in solution for automatically archiving, compressing, and cleaning up log files based on rules you define. This guide covers everything from basic setup to advanced configurations, common gotchas you’ll encounter, and how logrotate compares to other log management approaches.

How Logrotate Works Under the Hood

Logrotate operates through a combination of cron jobs and configuration files that define rotation policies. The main process runs daily via /etc/cron.daily/logrotate, checking all configured log files against their rotation criteria like size limits, age, or manual triggers.

The rotation process follows this sequence:

• Check if rotation criteria are met (size, time, or forced rotation)
• Execute pre-rotation scripts if defined
• Stop or signal the service writing to the log (optional)
• Move/rename the current log file
• Create a new empty log file with proper permissions
• Compress old log files if configured
• Remove old log files beyond the retention limit
• Execute post-rotation scripts
• Restart or signal the service to resume logging

Ubuntu 24 includes logrotate 3.21.0 by default, which introduced better handling of large files and improved error reporting compared to earlier versions.

Installation and Basic Setup

Logrotate comes pre-installed on Ubuntu 24, but let’s verify and check the setup:

# Check if logrotate is installed
dpkg -l | grep logrotate

# Check version
logrotate --version

# Install if missing (shouldn't be needed)
sudo apt update && sudo apt install logrotate

The default configuration structure includes:

• /etc/logrotate.conf – Main configuration file
• /etc/logrotate.d/ – Directory for service-specific configurations
• /var/lib/logrotate/status – Rotation status tracking
• /etc/cron.daily/logrotate – Daily cron job

Let’s examine the default global configuration:

# View main config
cat /etc/logrotate.conf

# Check existing service configs
ls -la /etc/logrotate.d/

# View rotation status
sudo cat /var/lib/logrotate/status

Creating Custom Log Rotation Configurations

The most common scenario is setting up rotation for application logs. Here’s a practical example for a web application:

# Create config for custom app logs
sudo nano /etc/logrotate.d/myapp

# Basic configuration
/var/log/myapp/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0644 www-data www-data
    postrotate
        systemctl reload nginx
    endscript
}

For high-traffic applications that generate large logs quickly, you might need size-based rotation:

# Size-based rotation for busy applications
/var/log/myapp/access.log {
    size 100M
    rotate 10
    compress
    delaycompress
    missingok
    notifempty
    create 0644 www-data www-data
    copytruncate
    postrotate
        /usr/bin/systemctl reload myapp
    endscript
}

Here’s a more complex example for database logs with custom compression and retention:

# Advanced MySQL log rotation
/var/log/mysql/*.log {
    daily
    rotate 90
    compress
    compresscmd /usr/bin/xz
    compressext .xz
    compressoptions -6
    delaycompress
    missingok
    notifempty
    create 0640 mysql adm
    sharedscripts
    prerotate
        /usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs
    endscript
    postrotate
        # Signal MySQL to reopen log files
        test -x /usr/bin/mysqladmin && \
        /usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs
    endscript
}

Configuration Options and Directives

Understanding the available directives is crucial for effective log management. Here’s a comprehensive breakdown:

Directive	Purpose	Example	Notes
daily/weekly/monthly	Rotation frequency	daily	Can also use specific times
size	Rotate when file reaches size	size 100M	Overrides time-based rotation
rotate	Number of old logs to keep	rotate 30	0 means delete immediately
compress	Compress rotated logs	compress	Uses gzip by default
copytruncate	Copy and truncate original	copytruncate	For apps that can’t reopen logs
create	Create new log file	create 0644 user group	Alternative to copytruncate
missingok	Don’t error if log missing	missingok	Useful for optional logs

Real-World Use Cases and Examples

Let’s look at some practical scenarios you’ll encounter in production environments.

High-Volume Web Server Logs

# Nginx access logs for high-traffic site
/var/log/nginx/access.log {
    hourly
    rotate 168  # Keep 1 week of hourly logs
    size 500M
    compress
    delaycompress
    missingok
    notifempty
    create 0644 www-data adm
    sharedscripts
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi
    endscript
    postrotate
        invoke-rc.d nginx rotate >/dev/null 2>&1
    endscript
}

Application Logs with Centralized Logging

For applications that send logs to centralized systems like ELK stack, you might want faster local cleanup:

# Fast cleanup for centrally logged apps
/var/log/app/{error,access,debug}.log {
    daily
    rotate 3  # Only keep 3 days locally
    size 50M
    compress
    missingok
    notifempty
    copytruncate
    postrotate
        # Ensure log shipper picks up rotated files
        systemctl restart filebeat
    endscript
}

System Service Logs

# Custom service with multiple log levels
/var/log/myservice/*.log {
    weekly
    rotate 52
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myservice syslog
    postrotate
        # Send SIGUSR1 to reopen logs
        /bin/kill -USR1 $(cat /run/myservice.pid 2>/dev/null) 2>/dev/null || true
    endscript
}

Testing and Debugging Configurations

Before deploying rotation configs to production, always test them thoroughly:

# Test specific configuration without executing
sudo logrotate -d /etc/logrotate.d/myapp

# Force rotation for testing (be careful in production)
sudo logrotate -f /etc/logrotate.d/myapp

# Verbose output for troubleshooting
sudo logrotate -v /etc/logrotate.conf

# Check what files would be affected
sudo logrotate -d /etc/logrotate.conf | grep "rotating pattern"

To test a configuration before the daily cron runs:

# Temporarily modify status file to force rotation
sudo cp /var/lib/logrotate/status /var/lib/logrotate/status.backup

# Remove specific log entry to force rotation
sudo sed -i '/myapp/d' /var/lib/logrotate/status

# Run logrotate manually
sudo logrotate -v /etc/logrotate.d/myapp

# Restore status file if needed
sudo cp /var/lib/logrotate/status.backup /var/lib/logrotate/status

Common Issues and Troubleshooting

Here are the most frequent problems you’ll encounter and how to solve them:

Permission Issues

# Common error: permission denied
# Check log file ownership
ls -la /var/log/myapp/

# Fix ownership if needed
sudo chown -R www-data:www-data /var/log/myapp/

# Verify logrotate can read the directory
sudo -u www-data ls /var/log/myapp/

Service Not Reopening Log Files

This is probably the most common issue. Services need to be told to reopen their log files after rotation:

# Wrong approach - file handle remains open
/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    # Missing postrotate script
}

# Correct approach - signal service to reopen
/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    postrotate
        systemctl reload myapp
        # Or send specific signal
        # kill -USR1 $(cat /run/myapp.pid)
    endscript
}

Logrotate Not Running

# Check if daily cron is working
sudo run-parts --test /etc/cron.daily

# Check logrotate cron job
cat /etc/cron.daily/logrotate

# Check system logs for errors
grep logrotate /var/log/syslog

# Check logrotate status
sudo systemctl status cron

Configuration Syntax Errors

# Validate configuration syntax
sudo logrotate -d /etc/logrotate.conf

# Common syntax issues:
# - Missing endscript after postrotate
# - Incorrect file permissions in create directive
# - Invalid size specifications (use M for MB, G for GB)
# - Duplicate directives in same config block

Performance Considerations and Optimization

Logrotate performance can impact system resources, especially with large files or many simultaneous rotations:

Scenario	File Size	Rotation Time	CPU Usage	Optimization
Small logs (<10MB)	1-10MB	<1 second	Low	Standard compression
Medium logs (10-100MB)	10-100MB	5-30 seconds	Medium	delaycompress
Large logs (100MB-1GB)	100MB-1GB	1-10 minutes	High	copytruncate + nice
Very large logs (>1GB)	>1GB	10+ minutes	Very High	Size-based rotation

For large files, consider using copytruncate to minimize service interruption:

# Optimized config for large files
/var/log/bigapp/massive.log {
    size 500M  # Rotate before files get too large
    rotate 20
    compress
    delaycompress
    copytruncate  # Avoids service restart
    missingok
    notifempty
    # Use nice to reduce CPU priority
    prerotate
        echo "Starting rotation of large log file"
    endscript
    postrotate
        # Optional: signal external monitoring
        echo "Large log rotated" | wall
    endscript
}

Advanced Configuration Techniques

Custom Compression Settings

# Use different compression algorithms
/var/log/archive/*.log {
    daily
    rotate 365
    compress
    compresscmd /usr/bin/xz
    compressext .xz
    compressoptions -9  # Maximum compression
    uncompresscmd /usr/bin/unxz
    delaycompress
    missingok
    notifempty
}

Conditional Rotation Based on Time

# Rotate only during off-peak hours
/var/log/peak-app/*.log {
    daily
    rotate 30
    compress
    missingok
    notifempty
    prerotate
        # Only rotate between 2-4 AM
        hour=$(date +%H)
        if [ $hour -lt 2 ] || [ $hour -gt 4 ]; then
            exit 1
        fi
    endscript
    postrotate
        systemctl reload peak-app
    endscript
}

Log Archival to Remote Storage

# Archive old logs to S3 or remote storage
/var/log/important/*.log {
    daily
    rotate 7  # Keep only 1 week locally
    compress
    delaycompress
    missingok
    notifempty
    create 0644 app app
    lastaction
        # Archive logs older than 7 days to remote storage
        find /var/log/important -name "*.log.*.gz" -mtime +7 -exec \
        aws s3 cp {} s3://my-log-bucket/$(hostname)/ \; -delete
    endscript
}

Comparison with Alternative Log Management Solutions

Solution	Best For	Pros	Cons	Resource Usage
logrotate	System logs, simple rotation	Built-in, reliable, lightweight	Limited features, cron-dependent	Very Low
rsyslog	Centralized logging, filtering	Real-time, powerful filtering	Complex configuration	Medium
journald	systemd services	Integrated with systemd, binary format	Not universal, requires systemd	Medium
Fluentd/Fluent Bit	Container environments, streaming	Real-time, many outputs	Resource intensive, complex	High

Best Practices and Security Considerations

Follow these practices to maintain secure and efficient log rotation:

• Always use the create directive with appropriate permissions rather than relying on defaults
• Set up log retention policies that comply with your organization’s requirements
• Use delaycompress to allow log analysis tools to finish processing files
• Monitor logrotate execution through system logs
• Test configurations in staging environments before production deployment
• Consider disk space carefully when setting rotation policies
• Use sharedscripts when multiple log files share the same service
• Implement proper error handling in pre/post rotation scripts

Security Configuration Example

# Secure log rotation with proper permissions
/var/log/secure/*.log {
    weekly
    rotate 52
    compress
    delaycompress
    missingok
    notifempty
    create 0600 root root  # Restrict access to sensitive logs
    sharedscripts
    prerotate
        # Ensure only root can read rotated security logs
        umask 077
    endscript
    postrotate
        # Log rotation event for audit purposes
        logger "Security logs rotated by logrotate"
        systemctl reload rsyslog
    endscript
}

Monitoring and Maintenance

Set up monitoring to ensure logrotate continues working properly:

# Create monitoring script
sudo nano /usr/local/bin/check-logrotate.sh

#!/bin/bash
# Check if logrotate ran successfully today
last_run=$(stat -c %Y /var/lib/logrotate/status)
current_time=$(date +%s)
hours_since=$((($current_time - $last_run) / 3600))

if [ $hours_since -gt 25 ]; then
    echo "WARNING: logrotate hasn't run in $hours_since hours"
    exit 1
fi

# Check for errors in syslog
if grep -q "logrotate.*error" /var/log/syslog; then
    echo "ERROR: logrotate errors found in syslog"
    exit 1
fi

echo "logrotate is working properly"
exit 0

For environments with VPS hosting or dedicated servers, consider setting up centralized log monitoring to track rotation status across multiple systems.

Regular maintenance tasks include:

• Reviewing disk usage trends to adjust retention policies
• Checking for failed rotations in system logs
• Updating configurations when adding new services
• Testing rotation configurations after system upgrades
• Monitoring compression ratios to optimize storage

With proper logrotate configuration, you can automate log management effectively while maintaining system performance and ensuring you have the log data you need for troubleshooting and compliance requirements.

---

---

This article incorporates information and material from various online sources. We acknowledge and appreciate the work of all original authors, publishers, and websites. While every effort has been made to appropriately credit the source material, any unintentional oversight or omission does not constitute a copyright infringement. All trademarks, logos, and images mentioned are the property of their respective owners. If you believe that any content used in this article infringes upon your copyright, please contact us immediately for review and prompt action.

This article is intended for informational and educational purposes only and does not infringe on the rights of the copyright owners. If any copyrighted material has been used without proper credit or in violation of copyright laws, it is unintentional and we will rectify it promptly upon notification. Please note that the republishing, redistribution, or reproduction of part or all of the contents in any form is prohibited without express written permission from the author and website owner. For permissions or further inquiries, please contact us.