How to Set Up an NFS Server Using Block Storage
Setting up an NFS (Network File System) server with block storage is a game-changer for developers and sysadmins managing distributed environments. While cloud storage solutions are everywhere, having your own NFS server gives you complete control over data access patterns, latency, and costs – especially when you’re dealing with high-performance applications or strict compliance requirements. In this guide, we’ll walk through building a robust NFS server using block storage, covering everything from initial setup to performance tuning and real-world gotchas that’ll save you hours of debugging.
How NFS with Block Storage Works
NFS operates as a distributed file system protocol that allows remote hosts to mount file systems over a network and interact with them as if they were mounted locally. When you combine NFS with block storage, you’re essentially creating a centralized storage solution where the NFS server manages file operations on top of raw block devices.
The magic happens at the storage layer. Block storage provides raw, unformatted storage volumes that appear as attached drives to your server. Unlike object storage, block storage offers consistent I/O performance and low latency, making it perfect for NFS workloads that require predictable performance patterns.
Here’s the typical architecture flow:
- Block storage volume attached to NFS server as /dev/sdb, /dev/sdc, etc.
- File system (ext4, xfs, or btrfs) created on the block device
- NFS daemon exports the mounted file system
- Client machines mount the NFS export and access files transparently
Step-by-Step NFS Server Setup
Let’s build this thing from scratch. I’m assuming you’re working with a Linux server – Ubuntu 20.04+ or CentOS 8+ work great for this setup.
Prerequisites and Initial Setup
First, make sure your server has adequate resources. For production NFS servers, you’ll want at least 4GB RAM and fast network connectivity. If you’re running this on a VPS or dedicated server, ensure block storage volumes are properly attached.
# Update system packages
sudo apt update && sudo apt upgrade -y
# Install NFS server packages
sudo apt install nfs-kernel-server nfs-common -y
# For CentOS/RHEL:
# sudo yum install nfs-utils -y
Block Storage Preparation
Assuming your block storage appears as /dev/sdb (check with lsblk), let’s prepare it:
# Check available block devices
lsblk
# Create partition table (if needed)
sudo fdisk /dev/sdb
# Type 'n' for new partition, accept defaults, then 'w' to write
# Create file system - XFS is excellent for NFS workloads
sudo mkfs.xfs -f /dev/sdb1
# Create mount point
sudo mkdir -p /nfs/storage
# Mount the volume
sudo mount /dev/sdb1 /nfs/storage
# Make mount persistent
echo '/dev/sdb1 /nfs/storage xfs defaults,noatime 0 0' | sudo tee -a /etc/fstab
NFS Server Configuration
Now for the NFS-specific setup. The /etc/exports file is where the magic happens:
# Create directory structure
sudo mkdir -p /nfs/storage/shared
sudo chown nobody:nogroup /nfs/storage/shared
sudo chmod 755 /nfs/storage/shared
# Configure NFS exports
sudo nano /etc/exports
# Add these lines to /etc/exports:
/nfs/storage/shared 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)
/nfs/storage/shared 10.0.0.0/8(rw,sync,no_subtree_check,no_root_squash)
Export options breakdown:
- rw: Read-write access
- sync: Write operations complete before responding (safer but slower)
- no_subtree_check: Disables subtree checking (better performance)
- no_root_squash: Allows root user access (use carefully)
Start and Enable Services
# Apply export configuration
sudo exportfs -arv
# Start NFS services
sudo systemctl start nfs-kernel-server
sudo systemctl enable nfs-kernel-server
# Verify exports
sudo exportfs -v
showmount -e localhost
Client-Side Configuration
Setting up clients is straightforward, but there are performance considerations:
# Install NFS client utilities
sudo apt install nfs-common -y
# Create mount point
sudo mkdir -p /mnt/nfs-share
# Mount with performance options
sudo mount -t nfs -o vers=4,proto=tcp,fsc 192.168.1.100:/nfs/storage/shared /mnt/nfs-share
# For persistent mounting, add to /etc/fstab:
192.168.1.100:/nfs/storage/shared /mnt/nfs-share nfs vers=4,proto=tcp,fsc 0 0
Real-World Use Cases and Examples
I’ve deployed NFS+block storage setups in several scenarios, and here are the ones that work particularly well:
Development Team Shared Storage
Perfect for teams needing shared access to large datasets, Docker registries, or build artifacts. The consistent performance of block storage means your CI/CD pipelines won’t randomly slow down due to storage I/O issues.
# Example: Docker registry storage backend
/nfs/storage/docker-registry 192.168.1.0/24(rw,sync,no_subtree_check)
# Mount on multiple Docker hosts
sudo mount -t nfs -o vers=4,proto=tcp,hard,timeo=600 \
nfs-server:/nfs/storage/docker-registry /var/lib/registry
Database Backup Storage
Block storage provides the consistent I/O performance needed for database backups, while NFS makes those backups accessible from multiple servers for validation or restore operations.
# Example backup script
#!/bin/bash
NFS_BACKUP_PATH="/mnt/nfs-backup/mysql"
mysqldump --all-databases --single-transaction \
--routines --triggers | gzip > "${NFS_BACKUP_PATH}/backup-$(date +%Y%m%d-%H%M%S).sql.gz"
Web Server Static Content
Serving static assets from NFS allows multiple web servers to share the same content without complex synchronization.
Performance Optimization and Tuning
Out-of-the-box NFS performance can be disappointing, but proper tuning makes a huge difference. Here’s what actually works:
| Parameter | Default | Recommended | Impact |
|---|---|---|---|
| rsize/wsize | 1048576 | 1048576 | Network efficiency |
| timeo | 600 | 150 | Faster failover |
| retrans | 2 | 3 | Network reliability |
| proto | tcp | tcp | Reliability over UDP |
Server-Side Tuning
# Increase NFS daemon threads
echo 'RPCNFSDCOUNT=16' >> /etc/default/nfs-kernel-server
# Tune kernel parameters
echo 'net.core.rmem_default = 262144' >> /etc/sysctl.conf
echo 'net.core.rmem_max = 16777216' >> /etc/sysctl.conf
echo 'net.core.wmem_default = 262144' >> /etc/sysctl.conf
echo 'net.core.wmem_max = 16777216' >> /etc/sysctl.conf
# Apply changes
sudo sysctl -p
sudo systemctl restart nfs-kernel-server
Client-Side Optimization
# High-performance mount options
sudo mount -t nfs -o vers=4,proto=tcp,fsc,rsize=1048576,wsize=1048576,hard,timeo=150,retrans=3 \
nfs-server:/nfs/storage/shared /mnt/nfs-share
Comparing NFS with Alternatives
| Solution | Setup Complexity | Performance | Scalability | Best Use Case |
|---|---|---|---|---|
| NFS + Block Storage | Medium | High | Medium | Shared development, backups |
| GlusterFS | High | Medium | High | Distributed applications |
| Ceph | Very High | High | Very High | Large-scale storage clusters |
| SSHFS | Low | Low | Low | Quick file access, development |
NFS with block storage hits the sweet spot for many scenarios – easier than distributed systems like Ceph, but more performant and reliable than simple solutions like SSHFS.
Common Issues and Troubleshooting
Every NFS setup has its quirks. Here are the issues I’ve encountered most frequently:
Permission Problems
The classic “access denied” issue usually stems from UID/GID mismatches:
# Check user mapping
cat /etc/idmapd.conf
# Force refresh of ID mappings
sudo nfsidmap -c
sudo systemctl restart nfs-idmapd
# Debug permissions
ls -la /mnt/nfs-share
id username
Stale File Handles
This happens when the server restarts or exports change while clients have active mounts:
# Force unmount stale handles
sudo umount -f /mnt/nfs-share
sudo umount -l /mnt/nfs-share # lazy unmount if force fails
# Clear client cache
echo 3 | sudo tee /proc/sys/vm/drop_caches
# Remount
sudo mount -t nfs nfs-server:/nfs/storage/shared /mnt/nfs-share
Performance Issues
Slow NFS usually comes down to network configuration or mount options:
# Test network performance
iperf3 -s # on server
iperf3 -c server-ip # on client
# Monitor NFS statistics
nfsstat -c # client stats
nfsstat -s # server stats
# Check for network errors
cat /proc/net/rpc/nfs
Security Best Practices
NFS security requires attention to several layers:
Network Security
# Configure firewall (UFW example)
sudo ufw allow from 192.168.1.0/24 to any port nfs
sudo ufw allow from 192.168.1.0/24 to any port 2049
sudo ufw allow from 192.168.1.0/24 to any port 111
# For iptables:
iptables -A INPUT -s 192.168.1.0/24 -p tcp --dport 2049 -j ACCEPT
iptables -A INPUT -s 192.168.1.0/24 -p tcp --dport 111 -j ACCEPT
Export Security
# Secure export options
/nfs/storage/shared 192.168.1.0/24(rw,sync,no_subtree_check,root_squash,secure)
# Use specific host IPs instead of subnets when possible
/nfs/storage/shared 192.168.1.10(rw,sync,no_subtree_check,root_squash)
/nfs/storage/shared 192.168.1.11(rw,sync,no_subtree_check,root_squash)
Monitoring and Maintenance
Keep your NFS setup healthy with regular monitoring:
# Create monitoring script
#!/bin/bash
# nfs-health-check.sh
echo "=== NFS Server Health Check ==="
echo "Active exports:"
exportfs -v
echo -e "\nNFS statistics:"
nfsstat -s | head -10
echo -e "\nDisk usage:"
df -h /nfs/storage
echo -e "\nActive connections:"
ss -tan | grep :2049
For production environments, consider integrating with monitoring solutions like Prometheus or Nagios. The Prometheus node_exporter includes NFS metrics out of the box.
Advanced Configuration Options
Once you have the basics working, these advanced options can improve reliability and performance:
High Availability Setup
While true NFS HA requires clustering software, you can improve availability with proper backup strategies:
# Automated backup script for block storage
#!/bin/bash
BACKUP_DEV="/dev/sdc" # Secondary block storage
SOURCE_PATH="/nfs/storage"
# Create snapshot (if your block storage supports it)
# lvcreate -L1G -s -n nfs-snapshot /dev/vg0/nfs-volume
# Sync to backup device
rsync -av --delete $SOURCE_PATH /backup/nfs/
NFSv4 with Kerberos Authentication
For environments requiring strong authentication, NFSv4 with Kerberos provides enterprise-grade security:
# Install Kerberos packages
sudo apt install krb5-user krb5-config
# Configure /etc/exports for Kerberos
/nfs/storage/shared gss/krb5p(rw,sync,no_subtree_check)
# Client mount with Kerberos
sudo mount -t nfs -o vers=4,sec=krb5p server:/nfs/storage/shared /mnt/secure-share
The combination of NFS and block storage creates a robust, performant shared storage solution that scales well for small to medium-sized deployments. While it requires more setup than cloud storage services, the control and performance benefits make it worthwhile for many use cases. The key is proper planning, security configuration, and ongoing monitoring to keep everything running smoothly.
For additional technical details, check the official NFS documentation and the Linux kernel NFS documentation.
This article incorporates information and material from various online sources. We acknowledge and appreciate the work of all original authors, publishers, and websites. While every effort has been made to appropriately credit the source material, any unintentional oversight or omission does not constitute a copyright infringement. All trademarks, logos, and images mentioned are the property of their respective owners. If you believe that any content used in this article infringes upon your copyright, please contact us immediately for review and prompt action.
This article is intended for informational and educational purposes only and does not infringe on the rights of the copyright owners. If any copyrighted material has been used without proper credit or in violation of copyright laws, it is unintentional and we will rectify it promptly upon notification. Please note that the republishing, redistribution, or reproduction of part or all of the contents in any form is prohibited without express written permission from the author and website owner. For permissions or further inquiries, please contact us.
