Last updated: February 5, 2025
# Filter files in Ruby: a practical guide
#### Kevin van Zonneveld
Co-founder · Amsterdam, The Netherlands · Show bio
[](https://x.com/kvz)[](https://github.com/kvz)
Efficient file filtering is essential for everything from log processing to media management. Ruby provides robust tools for file selection that go beyond simple pattern matching. Let's explore practical techniques that scale from basic to advanced scenarios.
## Requirements
This guide requires Ruby 2.0 or later. The examples presented have been verified using Ruby 3.2.3 and mime-types 3.6.0. For MIME type detection, you'll need the mime-types gem:
```ruby
# Add to your gemfile
gem 'mime-types', '~> 3.6'
# Or install directly
gem install mime-types
```
###### Note
The mime-types gem employs modified semantic versioning to track both API changes and registry data updates. For further details, refer to the[mime-types documentation](https://github.com/mime-types/ruby-mime-types#readme).
## Core filtering methods
Ruby's standard library offers several immediate solutions for file filtering:
```ruby
# Filter by extension
pdf_files = Dir.children('/docs').select { |f| File.extname(f) == '.pdf' }
# Filter by size (1MB threshold)
large_files = Dir.glob('*').select do |f|
begin
File.size(f) > 1_000_000
rescue Errno::ENOENT, Errno::EACCES => e
warn "Error accessing #{f}: #{e.message}"
false
end
end
# Filter by modification time (last 24 hours)
recent_files = Dir.glob('*').select do |f|
begin
File.mtime(f) > (Time.now - 86400)
rescue Errno::ENOENT, Errno::EACCES => e
warn "Error accessing #{f}: #{e.message}"
false
end
end
```
These methods use the `File` class utilities for quick checks without loading file contents.
## Advanced pattern matching with glob
Ruby's `Dir.glob` supports UNIX-style pattern matching with some Ruby-specific enhancements:
```ruby
# Match nested Markdown files
markdown_files = Dir.glob('**/*.md')
# Filter files modified in January 2024
jan_files = Dir.glob('*').grep(/(2024-01-\d{2})/)
# Combined size and type filter
big_images = Dir.glob('*.{jpg,png}').select do |f|
begin
File.size(f) > 500_000
rescue Errno::ENOENT, Errno::EACCES => e
warn "Error accessing #{f}: #{e.message}"
false
end
end
```
Use double star (`**`) for recursive directory traversal and brace expansion for multiple extensions.
## Mime type detection
For more reliable type checking than extensions alone, use the `mime-types` gem:
```ruby
require 'mime/types'
def media_files(dir)
Dir.children(dir).select do |f|
begin
mime = MIME::Types.type_for(f).first
mime&.media_type == 'image' || mime&.media_type == 'video'
rescue StandardError => e
warn "Error processing #{f}: #{e.message}"
false
end
end
end
# Usage:
visual_assets = media_files('/content/assets')
```
This approach checks the actual MIME type registry rather than relying solely on file extensions. For additional details, see the[mime-types gem documentation](https://github.com/mime-types/ruby-mime-types#readme).
## Metadata filtering
Combine multiple metadata points for precise selection:
```ruby
def recent_documents(path)
Dir.glob("#{path}/*").select do |f|
begin
next unless File.file?(f)
ext = File.extname(f).downcase
size = File.size(f)
modified = File.mtime(f)
(ext == '.pdf' || ext == '.docx') &&
size.between?(10_000, 5_000_000) &&
modified > (Time.now - 7*86400)
rescue StandardError => e
warn "Error processing #{f}: #{e.message}"
false
end
end
end
```
This selects PDF/DOCX files modified in the last week between 10 KB and 5 MB.
## Performance considerations
When processing large directories:
1. **Lazy Evaluation**: Use `lazy` with large result sets
```ruby
Dir.glob('**/*').lazy
.select { |f| File.size(f) > 1_000_000 }
.first(10)
```
2. **Early Exit**: Fail fast with `break` when possible
```ruby
Dir.children('/logs').each do |f|
begin
next unless f.end_with?('.log')
break if File.size(f) > 1_000_000_000 # Stop at first huge log
process_log(f)
rescue StandardError => e
warn "Error processing #{f}: #{e.message}"
end
end
```
3. **Metadata Caching**: Store frequently accessed data
```ruby
# Note: As of mime-types 3.6.0, caching is based on the data gem version instead of the mime-types version.
file_cache = {}
Dir.glob('*').each do |f|
begin
file_cache[f] = {
mtime: File.mtime(f),
size: File.size(f)
}
rescue StandardError => e
warn "Error caching #{f}: #{e.message}"
end
end
```
## Production-grade example
Here's a complete filtering module with error handling:
```ruby
require 'mime/types'
class FileFilter
def initialize(root_dir)
@root = root_dir
end
def find_files(extensions: [], min_size: 0, max_age: Float::INFINITY)
Dir.glob(File.join(@root, '**', '*')).lazy.select do |path|
begin
next unless File.file?(path)
valid_extension = extensions.empty? || extensions.include?(File.extname(path))
valid_size = File.size(path) >= min_size
valid_age = (Time.now - File.mtime(path)) < max_age
valid_extension && valid_size && valid_age
rescue StandardError => e
warn "Error processing #{path}: #{e.message}"
false
end
end
rescue Errno::ENOENT => e
warn "Directory error: #{e.message}"
[].lazy
end
end
# Usage:
filter = FileFilter.new('/user/uploads')
recent_images = filter.find_files(
extensions: ['.jpg', '.png'],
min_size: 100_000,
max_age: 3600 # 1 hour
).take(100)
```
These examples illustrate practical ways to filter files in Ruby with robust error handling. For more advanced file processing solutions, consider exploring Transloadit's API—visit our[documentation](/index.md) for additional details.
## Handling file encodings
Ruby typically handles file name encodings automatically when using UTF-8. However, if you encounter issues with non-ASCII characters in file names, you can enforce UTF-8 encoding as follows:
```ruby
# Convert file names to utf-8 encoding
utf8_files = Dir.glob('*').map { |f| f.force_encoding('UTF-8') }
```
This simple approach ensures that all file names are treated as UTF-8, minimizing potential encoding-related issues.
## Conclusion
Throughout this guide, we explored practical techniques to filter files in Ruby—from simple extension checks and glob patterns to MIME type detection and metadata filtering. By combining these methods, you can build efficient file processing pipelines tailored to your needs. For large-scale, production-ready solutions, consider exploring the robust file processing API offered by Transloadit. Visit our [documentation](/index.md) to learn more.
\#ruby#file-filtering-service#file-processing#glob-patterns
### 👩💻 Join 20k+ developers
Sign up for our [monthly newsletter](/newsletters.md) to receive direct links to 3 exclusive tech — and 2 product updates. No less, no more.
Your email:
Get access
## File uploading and encoding. Made simple.
Transloadit streamlines file handling for developers, trusted by brands like Coursera and The New York Times. We’re known for a reliable API, top-notch support, and a strong commitment to open source, with projects like [Uppy](https://uppy.io) and [Tus](https://tus.io) setting standards in file processing.
[Sign up](/c/)[Book a Demo](https://survey.typeform.com/to/kRg47Xi5)
No credit card needed · 5 GB included in the free plan
Cancel anytime