Cyber Services / All lessons / Python Programming / File and System Operations

File and System Operations

Module 22 of 31 · 3 min read · Free to read

MODULES ↓ Command blocks have a COPY button
Reading, Writing, and Managing Files with Python

Why File Operations Matter

File I/O (Input/Output) is essential for any real-world application: saving user data, processing logs, importing configurations, exporting reports, and more. Python provides simple yet powerful tools for working with files and the file system.

The open() Function

The core of file handling in Python is the built-in open() function, which returns a file object. Always specify the file path and mode.

Mode Description
'r'Read (default). File must exist.
'w'Write. Creates new file or overwrites existing.
'a'Append. Adds to end of file.
'x'Exclusive creation. Fails if file exists.
'b'Binary mode (e.g., 'rb', 'wb').
't'Text mode (default).
'+'Read and write (e.g., 'r+', 'w+').

Best Practice: Use Context Managers (with Statement)

Always use the with statement to ensure files are properly closed—even if an error occurs.

Reading a File Safely
# Read entire file
with open('data.txt', 'r') as file:
    content = file.read()
    print(content)

# Read line by line (memory efficient for large files)
with open('log.txt', 'r') as file:
    for line in file:
        print(line.strip())

Writing to Files

Writing and Appending
# Write (overwrites existing content)
with open('output.txt', 'w') as file:
    file.write("Hello, World!\n")
    file.write("This is a new line.")

# Append (adds to end)
with open('output.txt', 'a') as file:
    file.write("\nAppended line.")

Reading Methods Compared

  • file.read() → Returns entire file as a single string.
  • file.readline() → Returns next line (including \n).
  • file.readlines() → Returns list of all lines (loads entire file into memory).
When to Use Which Method
# Small file: readlines() is fine
with open('config.txt') as f:
    lines = f.readlines()  # ['key1=value\n', 'key2=42\n']

# Large file: iterate line-by-line (lazy)
with open('huge_dataset.csv') as f:
    for line in f:  # Memory efficient!
        process(line)

Working with Different File Types

Text Files (Default)

Handled as strings. Specify encoding (e.g., UTF-8) to avoid issues:

with open('unicode.txt', 'r', encoding='utf-8') as f:
    text = f.read()
Binary Files

Use 'rb' or 'wb' for images, executables, etc.:

# Copy an image
with open('photo.jpg', 'rb') as src:
    with open('copy.jpg', 'wb') as dst:
        dst.write(src.read())
JSON Files

Use the json module for structured data:

import json

# Write JSON
data = {"name": "Ali", "age": 28}
with open('user.json', 'w') as f:
    json.dump(data, f)

# Read JSON
with open('user.json', 'r') as f:
    user = json.load(f)

File System Operations with os and pathlib

Python provides modules to interact with the operating system:

Common File System Tasks
import os
from pathlib import Path

# Check if file exists
if os.path.exists('data.txt'):
    print("File found!")

# Using pathlib (modern, object-oriented)
p = Path('data.txt')
if p.exists():
    print(f"Size: {p.stat().st_size} bytes")

# List directory contents
for item in Path('.').iterdir():
    print(item.name)

# Create directory
Path('new_folder').mkdir(exist_ok=True)

# Delete file
if p.exists():
    p.unlink()

Error Handling in File Operations

Always handle potential exceptions like FileNotFoundError or PermissionError.

Robust File Reading
try:
    with open('config.txt', 'r') as f:
        config = f.read()
except FileNotFoundError:
    print("Configuration file not found. Using defaults.")
except PermissionError:
    print("Access denied. Check file permissions.")
except Exception as e:
    print(f"Unexpected error: {e}")

Temporary Files and Directories

Use the tempfile module for secure, auto-cleaned temporary files.

Safe Temporary File
import tempfile

# Creates file that auto-deletes when closed
with tempfile.NamedTemporaryFile(mode='w', delete=True) as tmp:
    tmp.write("Temporary data")
    tmp.flush()
    print(f"Temp file: {tmp.name}")
# File is automatically deleted here

Best Practices

  • ✅ Always use with open(...) for automatic resource cleanup.
  • ✅ Specify encoding explicitly (e.g., encoding='utf-8').
  • ✅ Prefer pathlib.Path over os.path for new code (more readable).
  • ✅ Handle exceptions—file operations can fail for many reasons.
  • ✅ For large files, read line-by-line instead of loading everything into memory.

Common Pitfalls

  • ❌ Forgetting to close files (avoided by using with).
  • ❌ Assuming file paths are portable (use Path for cross-platform compatibility).
  • ❌ Ignoring encoding—leads to UnicodeDecodeError on non-ASCII files.
  • ❌ Using readlines() on huge files—causes memory exhaustion.

Key Takeaways

  • File operations are foundational for persistent data handling.
  • The with statement ensures safe, automatic file closing.
  • Choose the right read/write method based on file size and use case.
  • Use pathlib for modern, clean file system interactions.
  • Always handle errors—file I/O is inherently unreliable.
Modules (31) ← Python Programming