Secure file transfer with SFTP in Lua

Tim Koschützki

Co-founder · Berlin, Germany ·

SFTP transfers files over SSH. LuaSocket’s socket.ftp implements plaintext FTP, a different protocol, and cannot fulfill an SFTP connection. This article keeps its published URL but replaces the earlier FTP examples with an actual SFTP transport: Lua-cURL backed by libcurl with SSH support.

Security considerations

Obtain the server’s public host key through an authenticated channel and provision a private known_hosts file before connecting. Do not accept an unknown key automatically. Use a dedicated read-only SSH account and key, restricted by the server to the intended import directory.

The client rejects path traversal in names and never evaluates filenames as commands. That lexical check cannot prevent the server from resolving a symlink outside a directory. Require a server-side chroot or equivalent restriction, with no links to sensitive content, before automating imports. Use a trusted server directory whose entry names contain no newlines: libcurl’s name-only listing is line-delimited and cannot represent those filenames unambiguously.

Setting up the SFTP transport

Use LuaJIT 2.1, Lua-cURL 0.3.13-1, LuaFileSystem 1.8.0-1, and a supported libcurl build with SFTP support. The example was tested with libcurl 8.21.0 and libssh2 1.11.1. A system curl executable may use a different library than the Lua module; check the library actually linked to Lua-cURL. LuaRocks must use the headers for your LuaJIT interpreter.

luarocks --lua-version=5.1 install Lua-cURL 0.3.13-1
luarocks --lua-version=5.1 install luafilesystem 1.8.0-1

If necessary, supply LuaRocks’ CURL_DIR setting for the libcurl installation with SSH support. See the Lua-cURL project, libcurl host-key verification, and name-only directory listings.

Connecting to an SFTP server

Save the following complete module as sftp_import.lua. Connection settings come from trusted deployment configuration. root is an absolute directory within the server’s restricted account. The public and private key files must belong to that account’s dedicated client identity.

local curl = require("lcurl.safe")
local lfs = require("lfs")
local M = {}

local function name_ok(name)
  return type(name) == "string" and #name <= 255 and name ~= "." and name ~= ".."
    and name:match("^[A-Za-z0-9][A-Za-z0-9._ -]*$") ~= nil
end

local function encode(path)
  return (path:gsub("([^A-Za-z0-9/_.~-])", function(byte)
    return string.format("%%%02X", byte:byte())
  end))
end

function M.remote_path(root, name)
  assert(type(root) == "string" and root:sub(1, 1) == "/", "Absolute remote root required")
  assert(not root:find("//", 1, true), "Invalid remote root")
  for part in root:gmatch("[^/]+") do assert(name_ok(part), "Invalid remote root component") end
  if name ~= nil then assert(name_ok(name), "Unsafe remote name") end
  local prefix = root:gsub("/+$", "") .. "/"
  return prefix .. (name or "")
end

local function transfer(config, path, listing, sink)
  assert(config.host:match("^[A-Za-z0-9.-]+$"), "Invalid SSH hostname")
  local port = config.port or 22
  assert(type(port) == "number" and port == math.floor(port) and port > 0 and port < 65536, "Invalid SSH port")
  local handle = assert(curl.easy({
    url = "sftp://" .. config.host .. ":" .. port .. encode(path),
    protocols = curl.PROTO_SFTP,
    proxy = "",
    username = assert(config.user),
    ssh_auth_types = curl.SSH_AUTH_PUBLICKEY,
    ssh_knownhosts = assert(config.known_hosts),
    ssh_private_keyfile = assert(config.private_key),
    ssh_public_keyfile = assert(config.public_key),
    connecttimeout = 10,
    timeout = 60,
    dirlistonly = listing,
    writefunction = sink
  }))
  local ok, result, err = pcall(function() return handle:perform() end)
  handle:close()
  if not ok or not result then return nil, "SFTP transfer failed" end
  return true
end

function M.list_directory(config)
  local chunks, length = {}, 0
  local ok, err = transfer(config, M.remote_path(config.root), true, function(chunk)
    length = length + #chunk
    if length > 1024 * 1024 then return 0 end
    chunks[#chunks + 1] = chunk
    return #chunk
  end)
  if not ok then return nil, err end
  local entries = {}
  for name in table.concat(chunks):gmatch("([^\n]+)") do
    if name ~= "." and name ~= ".." then
      if not name_ok(name) then return nil, "Unsupported directory entry" end
      entries[#entries + 1] = name
    end
  end
  table.sort(entries)
  return entries
end

function M.download(config, name, output_directory)
  local remote = M.remote_path(config.root, name)
  -- An exclusive directory prevents overwriting an existing file or following its symlink.
  assert(lfs.mkdir(output_directory), "Output directory must be new, beneath a private parent")
  local destination = output_directory .. "/" .. name
  local file = assert(io.open(destination, "wb"))
  local size = 0
  local called, ok, err = pcall(transfer, config, remote, false, function(chunk)
    size = size + #chunk
    if size > 64 * 1024 * 1024 then return 0 end
    if not file:write(chunk) then return 0 end
    return #chunk
  end)
  local closed = file:close()
  if not called or not ok or not closed then
    os.remove(destination)
    return nil, "Could not complete download"
  end
  return destination
end
return M

Listing files and directories

list_directory() requests SFTP directory entries, not a file retrieval. It returns names only; those names can refer to files or directories. It does not parse Unix ls -l output, assume that every entry is a file, or treat a successful listing as permission to download everything.

The restricted name policy accepts ordinary names with spaces, dots, underscores, and hyphens. It rejects slashes, backslashes, control characters, percent escapes, and . or ... Expand that policy only with matching encoding and containment tests. An unsupported listing entry makes list_directory() return nil and Unsupported directory entry, without returning partial results.

Downloading files from the SFTP server

Save this caller as import.lua. It lists the configured directory and downloads only the exact filename supplied by the operator. The output parent must already be private and owned by the application. The new child directory and its file inherit the restrictive umask shown below.

local sftp = require("sftp_import")
local config = {
  host = assert(os.getenv("SFTP_HOST")),
  port = tonumber(os.getenv("SFTP_PORT") or "22"),
  user = assert(os.getenv("SFTP_USER")),
  known_hosts = assert(os.getenv("SFTP_KNOWN_HOSTS")),
  private_key = assert(os.getenv("SFTP_PRIVATE_KEY")),
  public_key = assert(os.getenv("SFTP_PUBLIC_KEY")),
  root = assert(os.getenv("SFTP_ROOT"))
}
local wanted = assert(arg[1], "remote filename required")
local entries, err = sftp.list_directory(config)
assert(entries, err)
local found = false
for _, name in ipairs(entries) do if name == wanted then found = true end end
assert(found, "Requested entry is not in the directory")
local saved, failure = sftp.download(config, wanted, assert(arg[2], "new output directory required"))
assert(saved, failure)
print("Download complete")

umask 077
luajit import.lua 'report September.csv' private-imports/job-001

Downloads are bounded to 64 MiB. A directory entry, missing file, host-key mismatch, interrupted transfer, or size-limit failure must not be reported as a successful file download. Partial local files are removed after transfer failures. The private job directory remains for inspection.

Automating SFTP file imports

Call the same module from a scheduler using explicit filenames and a fresh job directory. Decide which entries are eligible in application code; name-only listings contain no file-type or symlink information. The listing and download are separate operations, so the server may change a file between them. Coordinate immutable files or atomic server-side publication with the sender.

Error handling and best practices

Fail closed on host-key mismatches. Retry only after classifying the operational failure; an authentication error is not a reason to disable host verification. Keep keys and logs private. The example uses library calls for transfers and directory creation, so shell metacharacters cannot turn filenames into commands. Server-side access restrictions remain essential.

Alternative approaches for secure transfers

OpenSSH’s sftp client is another option when invoked through an API that accepts an argument array. Do not interpolate a filename into os.execute() or io.popen(). HTTPS is a separate option only when the source exposes an authenticated HTTPS download service.

Conclusion

Use an SSH-capable library, verified host keys, and explicit remote-name rules for SFTP imports. LuaSocket’s FTP client is not an SFTP implementation. For managed imports, see Transloadit’s 🤖 /sftp/import Robot.