Skip to content
CSY102 Week 05 Beginner

Explore memory and isolation concepts with guided practice before moving to reading resources.

Computing Systems & OS

Track your progress through this week's content

Opening Framing: Boundaries Are Not Natural

In Week 2, you learned that processes are execution contexts. In Week 3, that identity is contextual. In Week 4, that data persists beyond identity. Now we ask: what keeps all of these separate? What prevents one process from reading another's memory, one user's data from leaking to another?

People often speak about "system boundaries" as if they exist the way walls exist in a building. But boundaries inside computers are not natural objects. They are rules.

A computer's components are physically connected. Electrical signals flow. Memory is reused. CPU time is shared. Devices are multiplexed.

If separation exists, it exists because the operating system enforces it.

This week teaches a mental model you must never forget:

Isolation is an enforced illusion.

1) What Is Memory, Really?

In casual talk, memory is "where a program keeps data while it runs." That description is true but insufficient for security reasoning.

Memory is the resource that makes execution real:

  • Code is loaded into memory
  • Variables and secrets exist in memory
  • Keys, tokens, credentials, and session state live in memory

If an attacker can read or influence memory they shouldn't, many higher-level security controls become irrelevant.

Mental model: If the filesystem is the system's long-term memory, RAM is its present tense. Compromise memory, and you compromise everything that's currently happening.

2) Why Isolation Matters

Modern systems run many processes at once. Some are yours. Some belong to other users. Some belong to the operating system itself.

Isolation answers the question:

How can the system allow sharing of hardware without allowing interference?

Security depends on the idea that:

  • Your process cannot read another process's memory
  • Your process cannot overwrite system memory
  • One process crashing should not collapse everything else

If isolation fails, "permissions" become theatre — because permissions are enforced by the OS, and the OS itself is implemented in memory.

Key insight: memory isolation is the foundation that makes all other security controls possible. If processes can read each other's memory, file permissions, network controls, and authentication all become meaningless.

3) Boundaries Are Policed, Not Guaranteed

Here is the uncomfortable point: the boundary between "allowed" and "not allowed" is not a physical barrier. It is a set of checks performed by the system.

Every check can fail:

  • Through bugs: buffer overflows, use-after-free, integer overflows
  • Through complexity: the more code enforcing boundaries, the more potential flaws
  • Through configuration: debugging features, shared memory segments, permissive settings
  • Through resource exhaustion: memory pressure can cause systems to behave unexpectedly
  • Through hardware: CPU vulnerabilities like Meltdown and Spectre bypass software isolation

This is why experienced defenders do not treat isolation as absolute. They treat it as a guarantee with conditions.

Mental model: isolation is a promise the system tries to keep, not a law of nature.

4) Preview: Isolation at Scale

The isolation concepts you've learned this week are foundational. They return in expanded form:

  • Week 6 (Services): services run in isolated contexts with their own identities
  • Week 8 (Networking): network isolation determines what can communicate with what
  • Week 11 (Virtualisation): VMs and containers are isolation taken to the extreme — entire operating systems separated from each other

The principle remains the same: isolation is enforced separation. Whether it's memory between processes, network segments between hosts, or entire operating systems between VMs — the question is always the same: what enforces the boundary, and what happens when it fails?

Key insight: Week 5 completes the foundation. Everything from Week 6 onward builds on processes, identity, persistence, and isolation. You now have the conceptual tools to analyze any system.

Real-World Context: When Isolation Fails

Memory isolation failures have caused some of the most severe vulnerabilities in computing history:

Meltdown and Spectre (2018): These CPU vulnerabilities allowed processes to read memory they shouldn't have access to — including kernel memory and other processes' data. The isolation enforced by the operating system was bypassed by exploiting how CPUs speculatively execute instructions. Every major processor vendor was affected. The "enforced illusion" of memory isolation was broken at the hardware level.

Heartbleed (2014): A bug in OpenSSL allowed attackers to read up to 64KB of server memory per request. That memory could contain private keys, passwords, session tokens — whatever happened to be in memory at the time. The boundary between "data I should see" and "data I shouldn't see" was violated through a simple bounds-checking error.

Buffer Overflow Exploits: Classic buffer overflows allow attackers to write beyond allocated memory boundaries, potentially overwriting return addresses to hijack execution. This entire class of vulnerability exists because memory boundaries are enforced by software checks that can be bypassed. MITRE ATT&CK documents exploitation for privilege escalation as technique T1068.

Common thread: isolation is only as strong as its enforcement. Hardware bugs, software bugs, and configuration errors can all break the boundaries we depend on. Defenders must assume boundaries can fail and design accordingly.

Guided Lab: Observing Memory and Contention

This lab stays conceptual: we are not exploiting memory. We are observing how memory behaves under normal pressure, and how resource competition can create instability and risk.

Lab Objective

Observe that memory is finite, shared, and managed — and that system stability depends on how memory pressure is handled.

Step 1: Observe memory state

Ask the system for a high-level view of memory usage:

free -h

You are not interpreting every number yet. You are noticing that the system tracks memory as a managed resource.

Step 2: Observe per-process memory usage

Ask the system which processes are using the most memory:

ps aux --sort=-%mem | head

Step 3: Induce light, safe memory pressure (optional)

If your VM has enough memory, you can safely create mild pressure by opening several browser tabs or running a few programs. The goal is not to crash the system — it is to observe change.

Re-run:

free -h

Observe what changed. Ask: what did the OS do to cope?

Reflection (mandatory)

  1. Why does memory pressure matter for security, even without attackers?
  2. What could go wrong if the system's memory management is stressed or buggy?
  3. Why is isolation best understood as an enforced promise?
  4. How does this week's concept of "enforced illusion" connect to Week 11's virtualisation topic?

Week 5 Outcome Check

By the end of this week, you should be able to explain:

Next week: System services, daemons, and the security problem of background trust. Week 5 completes the foundation — from Week 6 onward, we build on these concepts.

Weekly Reflection

Reflection Prompt (200-300 words):

This week introduced a foundational truth: isolation is an enforced illusion. The boundaries between processes, between users, between privilege levels — none of these are physical walls. They are rules enforced by software, and software can have bugs.

Reflect on the implications:

Connect this week to the bigger picture: Weeks 1-5 have built a foundation of how systems work. From Week 6 onward, we apply this knowledge to services, scheduling, networking, and more. How does understanding memory and isolation prepare you for those topics?

A strong response will explain why isolation failures are so severe, identify specific types of sensitive data that live in memory, and demonstrate understanding of how this foundational knowledge connects to upcoming topics.

🎯 Hands-On Labs (Free & Essential)

Explore memory and isolation concepts with guided practice before moving to reading resources.

🎮 TryHackMe: Linux Fundamentals Part 2

What you'll do: Inspect running processes and system resources to see how memory is allocated and controlled.
Why it matters: Memory isolation is enforced through process boundaries and OS controls.
Time estimate: 1.5-2 hours

Start TryHackMe Linux Fundamentals 2 →

📝 Lab Exercise: Memory Observation Checklist

Task: Use `top` or `htop` to identify the top 5 memory-consuming processes and note their owners.
Deliverable: Process list + short note on what happens if one is terminated.
Why it matters: Memory usage often reveals trust boundaries and risk concentration.
Time estimate: 45-60 minutes

🏁 PicoCTF Practice: Forensics (Strings + Memory)

What you'll do: Extract indicators from files using `strings` and basic analysis.
Why it matters: Memory and binary inspection often begins with simple extraction techniques.
Time estimate: 1-2 hours

Start PicoCTF Forensics →

🛡️ Lab: SUID Binary Exploit + Remediation

What you'll do: Identify a vulnerable SUID binary in a lab VM, exploit it safely, then remove or fix the risk.
Why it matters: Misconfigured SUID binaries are a common privilege escalation path.
Time estimate: 2-3 hours

Start GTFOBins Reference →

💡 Lab Tip: If two processes share data, ask what would happen if one is compromised.

🛡️ Secure Configuration & Privilege Boundaries

Memory isolation matters only if privilege boundaries are enforced. Secure configuration reduces escalation paths and prevents unsafe privilege inheritance.

Privilege hardening checklist:
- Audit SUID/SGID binaries regularly
- Enforce least privilege for services
- Restrict IPC channels (pipes, sockets)
- Apply resource limits to high-risk services

Isolation controls: Kernel protections like ASLR, DEP/NX, and sandboxing make exploitation harder but must be enabled and monitored.

📚 Building on CSY101 Week-13: Threat model privilege escalation paths before hardening. CSY201: Advanced OS hardening formalizes these controls for production systems.

Resources

Mark the required resources as complete to unlock the Week completion button.

Lab: Memory, Isolation, and System Boundaries

Goal: observe how memory is allocated, isolated, and protected, and why isolation failures affect the entire system.

Choose ONE path (Linux or Windows). Both are valid.

Linux Path (safe commands)

  1. Run free -h and record total, used, and available memory.
  2. Run top or htop (if available) and observe which processes are using the most memory.
  3. Pick one running process and note its memory usage.
  4. Conceptual observation: processes cannot normally see or read each other's memory. Explain why this matters.
  5. Concept question: What would it mean for system security if one process could freely read another's memory?

Windows Path (built-in tools)

  1. Open Task Manager → Performance → Memory. Record total and in-use memory.
  2. Go to the Processes tab and identify the top three memory-consuming processes.
  3. Observe that processes are isolated — one process cannot directly inspect another's memory.
  4. Concept question: Why does Windows treat memory isolation as a core security feature, not just a stability feature?

Deliverable (submit):

Checkpoint Questions

  1. Explain the difference between memory isolation and filesystem permissions.
  2. Why is memory considered a higher-risk shared resource than disk storage?
  3. What kinds of information might exist in memory that never appears on disk?
  4. Why do isolation failures (e.g., memory leaks across boundaries) have system-wide impact?
  5. How does this week prepare you to understand virtual machines and containers later in the unit?
  6. What does "isolation is an enforced illusion" mean? Give an example of how that illusion could break.

Verified Resources & Videos

Memory isolation is not about convenience. It is about preventing one component's compromise from instantly becoming everyone's compromise. MITRE ATT&CK shows how attackers exploit memory and boundary failures to escalate privileges.

← Previous: Week 04 Next: Week 06 →

Week 05 Quiz

Test your understanding of the weekly concepts.

Format: 10 multiple-choice questions. Passing score: 70%. Time: Untimed.

Take Quiz