Data Serialization: JSON, CSV, & Pickle

Serialization is the process of converting a Python object (like a dictionary or a list) into a format that can be stored on a disk or sent over a network. Deserialization is the reverse process.

In Python, we primarily work with three formats: JSON for web APIs, CSV for spreadsheets, and Pickle for internal Python-only storage.

---

1. JSON (JavaScript Object Notation)

JSON is the universal language of data on the web. It is text-based and readable by almost every programming language.

json_demo.py

import json

user = {"name": "Alice", "id": 101, "admin": True}

# Serialize to a string
json_str = json.dumps(user, indent=4)

# Serialize directly to a file
with open("user.json", "w") as f:
    json.dump(user, f)

JSON Limitations

JSON can only store basic types: strings, numbers, booleans, lists, and dicts. It cannot store complex Python objects like custom classes, sets, or datetime objects.

---

2. CSV (Comma Separated Values)

CSV is the standard for tabular data (Excel, Google Sheets).

csv_usage.py

import csv

data = [
    {"name": "Alice", "score": 95},
    {"name": "Bob", "score": 88}
]

# Using DictWriter is the most professional way to handle CSVs
with open("scores.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "score"])
    writer.writeheader()
    writer.writerows(data)

---

3. Pickle (The Python Native Format)

pickle is Python’s built-in serialization for complex objects. It can save almost anything, including custom class instances.

pickle_demo.py

import pickle

data = {"complex": {1, 2, 3}, "math": 3.14}

with open("data.pkl", "wb") as f: # Must use Binary mode 'wb'
    pickle.dump(data, f)

with open("data.pkl", "rb") as f:
    loaded = pickle.load(f)

Security Warning

NEVER unpickle data from an untrusted source. The pickle module can execute arbitrary code during the unpickling process. A malicious pickle file can compromise your entire computer. Use JSON or XML for data received from the internet.

---

4. Under the Hood: The Bytestream

When you serialize data, Python is converting high-level objects into a Bytestream (a sequence of 0s and 1s).

• JSON encodes this into UTF-8 text.
• Pickle encodes this into a proprietary binary format that is much more compact but only readable by Python.

---

5. Summary Table: Choosing a Format

Format	Readability	Speed	Compatibility	Best Use Case
JSON	High (Text)	Medium	Universal	Web APIs, Config files.
CSV	High (Text)	High	Excel/Data Science	Large tables of simple data.
Pickle	Low (Binary)	Very High	Python Only	Saving complex app state.
XML	Medium	Low	Universal	Legacy systems, Enterprise.