When working with files you often need to loop through a directory to read, filter, or process files.
# Example directory: /path/to/my_folder # Contains: file1.txt, image.png, data.csv, subdir/, etc.
Recommended approach (short answer)
Use pathlib for readability and cross-platform paths; use pathlib.rglob() or pathlib.glob() for pattern-based iteration. For maximum performance when you need filesystem metadata, use os.scandir() (or methods that use it under-the-hood, like os.walk() in modern Python).
Method 1: Modern & Readable — pathlib.Path (recommended)
from pathlib import Path
dir_path = Path("/path/to/my_folder")
# Non-recursive: direct children
for path in dir_path.iterdir():
if path.is_file():
print(path)
# Recursive: all files under dir_path
for path in dir_path.rglob("*"):
if path.is_file():
print(path)
# Pattern: only .txt files (non-recursive)
for path in dir_path.glob("*.txt"):
print(path)
Output:
/path/to/my_folder/file1.txt /path/to/my_folder/image.png /path/to/my_folder/data.csv /path/to/my_folder/subdir/file_inside.txt
Why use this?
> pathlib is object-oriented and cross-platform — your code is cleaner and less error-prone.
> .rglob() is concise for recursion and .glob() supports patterns.
Method 2: Fast iteration with os.scandir() (high performance)
import os
dir_path = "/path/to/my_folder"
with os.scandir(dir_path) as it:
for entry in it:
# entry is os.DirEntry — low-cost metadata access
if entry.is_file():
print(entry.path)
Output:
/path/to/my_folder/file1.txt /path/to/my_folder/image.png /path/to_my_folder/data.csv
os.scandir() returns DirEntry objects and avoids extra system calls (stat()), making it faster than os.listdir() when you need metadata. Use it when performance matters.
Method 3: Pattern matching with glob / lazy iglob
import glob import os
dir_path = "/path/to/my_folder"
# All files (returns list)
for fname in glob.glob(os.path.join(dir_path, "*")):
if os.path.isfile(fname):
print(fname)
# Lazy iterator (memory-friendly)
for fname in glob.iglob(os.path.join(dir_path, "**", "*.txt"), recursive=True):
print(fname)
Output:
/path/to_my_folder/file1.txt /path_to_my_folder/data.csv /path_to_my_folder/subdir/file_inside.txt
Use glob for shell-style patterns (*.txt, data_*.csv). Use iglob for lazy iteration on large directories to avoid big lists.
Method 4: Recursive traversal with os.walk()
import os
dir_path = "/path/to/my_folder"
for root, dirs, files in os.walk(dir_path):
for name in files:
print(os.path.join(root, name))
Output:
/path/to_my_folder/file1.txt /path/to_my_folder/image.png /path/to_my_folder/data.csv /path/to_my_folder/subdir/file_inside.txt
os.walk() is simple and returns the directory context (root, dirs, files). It’s useful if you need to prune directories, skip specific subfolders, or compute per-folder aggregates.
Note: modern os.walk() implementations use scandir() internally for better performance.
Method 5: Low-level & static analysis — using os.listdir()
import os
dir_path = "/path/to/my_folder"
for entry in os.listdir(dir_path):
full = os.path.join(dir_path, entry)
if os.path.isfile(full):
print(full)
Output:
/path/to_my_folder/file1.txt /path/to_my_folder/image.png /path/to_my_folder/data.csv
Simple and available across Python versions. But it returns names (strings) only and requires extra isfile() checks; slower than scandir() for metadata-heavy tasks.
Filtering by extension, name, or condition
from pathlib import Path
# Only .txt files recursively
for p in Path("/path/to/my_folder").rglob("*.txt"):
print(p)
# Using scandir and fnmatch for custom matching
import fnmatch, os
for entry in os.scandir("/path/to/my_folder"):
if entry.is_file() and fnmatch.fnmatch(entry.name, "data_*.csv"):
print(entry.path)
Combine pattern matching with directory iteration to focus only on the files you care about (faster, less I/O).
Error handling & permission-safe iteration
from pathlib import Path
dir_path = Path("/path/to/my_folder")
try:
for p in dir_path.iterdir():
try:
if p.is_file():
print(p)
except PermissionError:
print("Permission denied for", p)
except FileNotFoundError:
print(dir_path, "does not exist")
Performance tips & best practices
- Prefer pathlib for readable code, os.scandir() when you need speed and metadata. * os.walk() is ideal for full-tree traversal; glob.iglob() / Path.rglob() are great for pattern-based recursion.
- Use absolute paths or Path.resolve() to avoid surprises.
- Use lazy iterators (iglob, scandir iterator, Path.rglob) for very large directories.
- Avoid building giant lists when you can process files in a streaming fashion.
- When you need file metadata (size, mtime) prefer DirEntry.stat() or Path.stat() — they avoid extra system calls when available.
FAQs — Python: How to Loop Through Files in a Directory
How to loop through all files in a directory using Python?
Use the os module’s listdir() or scandir() to iterate through file names in a directory:
import os
for file in os.listdir('path/to/folder'):
print(file)This prints all files and subfolders in the specified directory.
How to loop through all files in a directory and its subdirectories in Python?
Use os.walk() to recursively loop through all folders and files:
import os
for root, dirs, files in os.walk('path/to/folder'):
for name in files:
print(os.path.join(root, name))This approach traverses the directory tree recursively.
How to loop through files in a directory using Python pathlib?
The modern pathlib module offers an easy and clean way to iterate through files:
from pathlib import Path
for file in Path('path/to/folder').iterdir():
print(file.name)Use iterdir() for direct children or rglob('*') for recursive traversal.
How to loop only through specific file types (like .txt or .csv) in a Python directory?
Use a file extension filter with endswith() or glob pattern matching:
import os
for file in os.listdir('path'):
if file.endswith('.txt'):
print(file)Or with pathlib:
from pathlib import Path
for file in Path('path').rglob('*.txt'):
print(file)How to skip directories while looping through files in Python?
Use os.path.isfile() to ensure you only process files:
import os
folder = 'path'
for file in os.listdir(folder):
full_path = os.path.join(folder, file)
if os.path.isfile(full_path):
print(file)How to loop through large directories efficiently in Python?
Use os.scandir() — it yields directory entries lazily without loading all names into memory:
import os
with os.scandir('path') as entries:
for entry in entries:
if entry.is_file():
print(entry.name)How to process each file while looping through a directory in Python?
You can perform operations like reading or renaming inside the loop:
import os
for file in os.listdir('path'):
with open(os.path.join('path', file)) as f:
data = f.read()
print(len(data))How to handle errors when looping through files in a Python directory?
Wrap your loop in a try-except block to handle permission or missing file errors:
import os
for file in os.listdir('path'):
try:
print(file)
except Exception as e:
print("Error:", e)