14. Security & Ethics

⚖️ The Legal Landscape

Computer Fraud and Abuse Act (CFAA — USA): The CFAA criminalizes accessing computer systems without authorization. The key phrase 'without authorization' has been contentiously interpreted. The landmark HiQ v. LinkedIn (2022) ruling held that accessing publicly available data does not violate the CFAA. However, accessing data behind a login you don't own, bypassing technical access controls (like CAPTCHAs implemented to block access), or accessing data after being sent a cease-and-desist can cross the CFAA line.

Computer Misuse Act (CMA — UK): Similar to CFAA. Unauthorized access to computer material carries criminal penalties. 'Unauthorized' is broadly interpreted.

GDPR (EU General Data Protection Regulation): Applies to any personal data of EU residents regardless of where your scraper runs. Scraping names, emails, phone numbers, or any identifiable information requires a legal basis. Most scraping use cases cannot satisfy GDPR's consent requirements. You cannot sell or store EU personal data without a lawful basis.

Terms of Service: Violating ToS is primarily a contractual issue, not criminal. However, continued access after a ToS violation cease-and-desist can transform ToS violation into unauthorized access under CFAA. LinkedIn, Twitter/X, and many others actively enforce ToS via litigation.

Copyright: Website content is automatically copyrighted. Extracting and republishing content (not just data/facts) requires either a license or fair use justification. Facts themselves are not copyrightable — price data, public statistics, and factual information can generally be freely collected.

✅ The Responsible Automation Checklist

Before deploying any automation, run through this checklist:

• Is the data publicly accessible? No authentication required to view it in a browser? If yes, generally safer territory.
• Does the site's robots.txt disallow it? Check programmatically. If disallowed, find an API or licensed data alternative.
• Have you checked the Terms of Service? Look for 'scraping', 'automated access', 'bots' provisions. Some ToS explicitly permit certain scraping.
• Is any personal data (PII) involved? Names, emails, phone numbers, addresses — these trigger GDPR/CCPA requirements. Don't collect unless you have a clear lawful basis and a plan for data security and deletion.
• Are you rate-limiting appropriately? Could your scraping materially degrade the target service's performance? If yes, reduce rate or use caching.
• Do you have a cease-and-desist plan? If the site operator contacts you demanding you stop, what is your response protocol?
• Is your automation for legitimate purposes? Price comparison, academic research, business intelligence, competitive analysis — generally legitimate. Fake account creation, spam delivery, vote manipulation, credential stuffing — illegal and harmful.

🔒 Operational Security for Automation Engineers

import os
from pathlib import Path
from cryptography.fernet import Fernet
import getpass

# NEVER DO THIS:
API_KEY = 'sk-proj-hardcoded-key-here'  # hardcoded in source code

# ALWAYS DO THIS:
API_KEY = os.environ.get('OPENAI_API_KEY')  # from environment
if not API_KEY:
    raise EnvironmentError('OPENAI_API_KEY environment variable is not set')

# For credentials that must be stored at rest:
class CredentialStore:
    def __init__(self, store_path: Path):
        self.store_path = store_path
        key_path = store_path.parent / '.key'
        if key_path.exists():
            self._key = key_path.read_bytes()
        else:
            self._key = Fernet.generate_key()
            key_path.write_bytes(self._key)
            key_path.chmod(0o600)  # owner read/write only
        self._fernet = Fernet(self._key)
    
    def store(self, name: str, value: str):
        encrypted = self._fernet.encrypt(value.encode())
        existing = self._load()
        existing[name] = encrypted.decode()
        import json
        self.store_path.write_text(json.dumps(existing))
    
    def retrieve(self, name: str) -> str:
        data = self._load()
        if name not in data:
            raise KeyError(f'Credential {name} not found')
        return self._fernet.decrypt(data[name].encode()).decode()
    
    def _load(self) -> dict:
        import json
        if self.store_path.exists():
            return json.loads(self.store_path.read_text())
        return {}

# Git hygiene for automation projects
# .gitignore must include:
# .env
# *.key
# credentials.json
# cookies*.json
# session*.json
# output/
# *.log

🧑‍💻 Data Minimization and Retention

Collect only what you need. Store it only as long as you need it. This is not just an ethical principle — it is a legal requirement under GDPR and a risk management strategy. A breach of a database containing data you didn't need to collect is a breach that didn't have to happen.

from datetime import datetime, timedelta
import sqlite3

def purge_old_records(db_path: str, table: str, date_column: str, retention_days: int = 90):
    '''Automatically purge records older than retention_days'''
    cutoff = (datetime.now() - timedelta(days=retention_days)).isoformat()
    with sqlite3.connect(db_path) as conn:
        cursor = conn.execute(
            f'DELETE FROM {table} WHERE {date_column} < ?', (cutoff,)
        )
        purged = cursor.rowcount
        conn.commit()
    print(f'Purged {purged} records older than {retention_days} days from {table}')
    return purged

# Run as part of daily maintenance
purge_old_records('automation.db', 'scraped_prices', 'scraped_at', retention_days=30)
purge_old_records('automation.db', 'user_events', 'created_at', retention_days=7)