Secure file transfer with SFTP in Lua
Use Lua-cURL with an SSH-enabled libcurl to list an SFTP directory and download one named file.
The example verifies the server’s host key and saves binary or empty files without invoking a shell
for transfers. LuaSocket’s
socket.ftp implements plaintext FTP,
a different protocol from SFTP.
Security considerations
Start with an existing SFTP account whose administrator supplies the hostname, port, username, and the directory as it appears inside the account’s chroot. The sender must place the file there before you run the importer. Use a dedicated read-only account with your client’s public key registered, and a private, unencrypted client key for this unattended example.
Obtain the server’s public host key through an authenticated channel and provision a private
OpenSSH-format known_hosts file. A scan alone does not establish trust. libcurl’s
host-key verification rejects unknown
or mismatched keys when that file is set.
The client rejects path traversal in names and never evaluates filenames as commands. That lexical check cannot prevent the server from resolving a symlink outside a directory. Require a server-side chroot containing only the intended data; a chroot limits access to its entire tree, not just the configured subdirectory. Use a trusted server directory whose entry names contain no newlines: libcurl’s name-only listing is line-delimited and cannot represent those filenames unambiguously.
Setting up the SFTP transport
The setup below targets Linux with Bash, LuaJIT 2.1, LuaRocks, a C compiler, make, curl,
unzip, and pkg-config. Install LuaJIT and libcurl development files first. pkg-config must find
both luajit and a libcurl built with libssh2 and SFTP support; their shared libraries must be
available to the dynamic loader. For a custom libcurl prefix, configure PKG_CONFIG_PATH and the
library loader for that installation before running setup.
Tested versions are LuaJIT 2.1, LuaRocks 3.8 and 3.13, Lua-cURL 0.3.13-1,
LuaFileSystem 1.8.0-1, libcurl 8.21.0 and 8.22.0, and libssh2 1.11.1.
These versions record the tested setups; keep your distribution’s security updates.
Lua-cURL’s pinned release is an older binding;
check compatibility when changing it. The setup selects development paths through pkg-config
rather than assuming that every header lives directly under /usr/include.
Save this as setup.sh in the parent where you want a new lua-sftp directory. It installs the
pinned Lua-cURL rock and LuaFileSystem
into that directory rather than a global LuaRocks tree.
#!/usr/bin/env bash
set -euo pipefail
unset LUA_PATH LUA_CPATH
for tool in luajit luarocks cc make curl unzip pkg-config; do
command -v "$tool" >/dev/null || { printf 'Missing prerequisite: %s\n' "$tool" >&2; exit 1; }
done
pkg-config --print-errors --exists luajit libcurl
lua_binary=$(command -v luajit)
c_compiler=$(command -v cc)
lua_include=$(pkg-config --variable=includedir luajit)
curl_include=$(pkg-config --variable=includedir libcurl)
curl_library=$(pkg-config --variable=libdir libcurl)
umask 077
mkdir lua-sftp
cd lua-sftp
curl -fsSLo lua-curl-0.3.13-1.src.rock https://luarocks.org/lua-curl-0.3.13-1.src.rock
curl -fsSLo luafilesystem-1.8.0-1.src.rock https://luarocks.org/luafilesystem-1.8.0-1.src.rock
luarocks --lua-version=5.1 --tree="$PWD/rocks" install \
lua-curl-0.3.13-1.src.rock \
LUA="$lua_binary" LUA_INCDIR="$lua_include" CC="$c_compiler" \
CURL_INCDIR="$curl_include" CURL_LIBDIR="$curl_library"
luarocks --lua-version=5.1 --tree="$PWD/rocks" install \
luafilesystem-1.8.0-1.src.rock \
LUA="$lua_binary" LUA_INCDIR="$lua_include" CC="$c_compiler"
mkdir private-imports
LUA_PATH='./rocks/share/lua/5.1/?.lua;./rocks/share/lua/5.1/?/init.lua' \
LUA_CPATH='./rocks/lib/lua/5.1/?.so' \
luajit -e 'local c=require("lcurl.safe"); print(c.version()); assert(c.version_info("protocols").SFTP, "libcurl lacks SFTP support"); assert(c.version():find("libssh2/", 1, true), "Use libcurl built with libssh2")'
bash setup.sh
An existing lua-sftp directory stops setup before installation. If installation fails, inspect the
error and retry the script from a fresh parent directory after fixing the prerequisite. The failed
directory remains for inspection. The final probe reports the libcurl used by Lua-cURL and checks
SFTP support; a system curl executable can use a different library.
Connecting to an SFTP server
Save the following complete module as lua-sftp/sftp_import.lua. Connection settings come from
trusted configuration. root is an absolute SFTP-visible directory, such as /exports, rather than
the server’s host filesystem path. The public and private client key files must be a matching pair.
local curl = require("lcurl.safe")
local lfs = require("lfs")
local M = {}
local function name_ok(name)
return type(name) == "string" and #name <= 255 and name ~= "." and name ~= ".."
and name:match("^[A-Za-z0-9][A-Za-z0-9._ -]*$") ~= nil
end
local function encode(path)
return (path:gsub("([^A-Za-z0-9/_.~-])", function(byte)
return string.format("%%%02X", byte:byte())
end))
end
function M.remote_path(root, name)
assert(type(root) == "string" and root:sub(1, 1) == "/", "Absolute remote root required")
assert(not root:find("//", 1, true), "Invalid remote root")
for part in root:gmatch("[^/]+") do assert(name_ok(part), "Invalid remote root component") end
if name ~= nil then assert(name_ok(name), "Unsafe remote name") end
local prefix = root:gsub("/+$", "") .. "/"
return prefix .. (name or "")
end
local function transfer(config, path, listing, sink)
assert(config.host:match("^[A-Za-z0-9.-]+$"), "Invalid SSH hostname")
local port = config.port or 22
assert(type(port) == "number" and port == math.floor(port) and port > 0 and port < 65536, "Invalid SSH port")
local handle = assert(curl.easy({
url = "sftp://" .. config.host .. ":" .. port .. encode(path),
protocols = curl.PROTO_SFTP,
proxy = "",
username = assert(config.user),
ssh_auth_types = curl.SSH_AUTH_PUBLICKEY,
ssh_knownhosts = assert(config.known_hosts),
ssh_private_keyfile = assert(config.private_key),
ssh_public_keyfile = assert(config.public_key),
connecttimeout = 10,
timeout = 60,
dirlistonly = listing,
writefunction = sink
}))
local ok, result = pcall(function() return handle:perform() end)
handle:close()
if not ok or not result then return nil, "SFTP transfer failed" end
return true
end
function M.list_directory(config)
local chunks, length = {}, 0
local ok, err = transfer(config, M.remote_path(config.root), true, function(chunk)
length = length + #chunk
if length > 1024 * 1024 then return 0 end
chunks[#chunks + 1] = chunk
return #chunk
end)
if not ok then return nil, err end
local entries = {}
for name in table.concat(chunks):gmatch("([^\n]+)") do
if name ~= "." and name ~= ".." then
if not name_ok(name) then return nil, "Unsupported directory entry" end
entries[#entries + 1] = name
end
end
table.sort(entries)
return entries
end
function M.download(config, name, output_directory)
local remote = M.remote_path(config.root, name)
-- An exclusive directory prevents overwriting an existing file or following its symlink.
assert(lfs.mkdir(output_directory), "Output directory must be new, beneath a private parent")
local destination = output_directory .. "/" .. name
local temporary = output_directory .. "/.download.part"
local file = io.open(temporary, "wb")
if not file then return nil, "Could not create download" end
local size = 0
local called, ok, err = pcall(transfer, config, remote, false, function(chunk)
size = size + #chunk
if size > 64 * 1024 * 1024 then return 0 end
if not file:write(chunk) then return 0 end
return #chunk
end)
local closed = file:close()
if not called or not ok or not closed then
os.remove(temporary)
return nil, "Could not complete download"
end
if not os.rename(temporary, destination) then
os.remove(temporary)
return nil, "Could not publish download"
end
return destination
end
return M
Listing files and directories
list_directory() uses libcurl’s
name-only directory listing. It returns names only;
those names can refer to files or directories. It does not parse Unix ls -l output, assume that
every entry is a file, or treat a successful listing as permission to download everything.
The restricted name policy accepts ASCII names with spaces, dots, underscores, and hyphens.
It rejects slashes, backslashes, control characters, percent escapes, and . or ... Expand that
policy only with matching encoding and containment tests. An unsupported listing entry makes
list_directory() return nil and Unsupported directory entry, without returning partial results.
Downloading files from the SFTP server
Save this caller as lua-sftp/import.lua. It lists the configured directory and downloads only the exact
filename supplied by the operator. The output parent must already be private and owned by the
application. The new child directory and its file inherit the restrictive umask shown below.
local sftp = require("sftp_import")
local config = {
host = assert(os.getenv("SFTP_HOST")),
port = tonumber(os.getenv("SFTP_PORT") or "22"),
user = assert(os.getenv("SFTP_USER")),
known_hosts = assert(os.getenv("SFTP_KNOWN_HOSTS")),
private_key = assert(os.getenv("SFTP_PRIVATE_KEY")),
public_key = assert(os.getenv("SFTP_PUBLIC_KEY")),
root = assert(os.getenv("SFTP_ROOT"))
}
local wanted = assert(arg[1], "remote filename required")
local entries, err = sftp.list_directory(config)
assert(entries, err)
local found = false
for _, name in ipairs(entries) do
print(name)
if name == wanted then found = true end
end
assert(found, "Requested entry is not in the directory")
local saved, failure = sftp.download(config, wanted, assert(arg[2], "new output directory required"))
assert(saved, failure)
print("Download complete")
(
cd lua-sftp || exit 1
umask 077
export SFTP_HOST='sftp.example.com' SFTP_PORT='22' SFTP_USER='import-reader'
export SFTP_ROOT='/exports'
export SFTP_KNOWN_HOSTS='/path/to/verified_known_hosts'
export SFTP_PRIVATE_KEY='/path/to/import-key' SFTP_PUBLIC_KEY='/path/to/import-key.pub'
LUA_PATH='./rocks/share/lua/5.1/?.lua;./rocks/share/lua/5.1/?/init.lua;./?.lua' \
LUA_CPATH='./rocks/lib/lua/5.1/?.so' \
luajit import.lua 'report September.csv' private-imports/job-001
)
Replace the connection values and key paths with those supplied for your account. Successful output
lists the directory’s names and ends with Download complete; the file is then available at
lua-sftp/private-imports/job-001/report September.csv. Compare its bytes or checksum with the
sender’s original to verify the result. A zero-byte file is a valid successful download.
The listing is limited to 1 MiB of received names, and each download to 64 MiB. Each operation has a 10-second connection limit and a 60-second total deadline, including connection time. A slow transfer can therefore fail below the size limit. A denied file, directory, dropped connection, or exceeded limit returns failure and removes the staging file. The new job directory remains; use a fresh job name when retrying. An existing job directory is refused without replacing its files.
The final filename appears only after the transfer succeeds and the file closes. Interrupting or
killing the process before publication can leave .download.part, but no final filename. Inspect or
remove that staging file after confirming the job has stopped; consumers should use only the returned final
path. This publication step does not guarantee storage durability across a machine crash.
Automating SFTP file imports
Call the same module from a scheduler using explicit filenames and a fresh job directory. Decide which entries are eligible in application code; name-only listings contain no file-type or symlink information. The listing and download are separate operations, so the server may change a file between them. Coordinate immutable files or atomic server-side publication with the sender.
Error handling and best practices
The caller exits with a nonzero status on failure. Check account settings and registered client keys
for authentication failures; resolve a host-key change with the administrator before updating
known_hosts. For local write failures, check free space and permissions on the private output
parent. Keep keys and logs private, and retry downloads in a fresh job directory.
Alternative approaches for secure transfers
OpenSSH’s sftp client is another option when invoked through an API that accepts an argument
array. Do not interpolate a filename into os.execute() or io.popen(). HTTPS is a separate option
only when the source exposes an authenticated HTTPS download service.
For managed imports, see Transloadit’s 🤖 /sftp/import Robot.