Opening Framing: Action Under Pressure
Last week covered incident response foundations. Now we focus on the critical middle phases: containment, eradication, and recovery. These are the hours and days where decisions directly impact whether the incident gets better or worse.
Containment is often a race against the attacker. While you're investigating, they may be expanding access, exfiltrating data, or deploying ransomware. Speed matters—but so does not making things worse.
This week covers containment strategies, eradication techniques, and the path to recovery. You'll learn to make sound decisions under pressure.
Key insight: Containment decisions involve tradeoffs. Isolating a system stops the attacker but may impact business operations. These tradeoffs require judgment, not just technical skill.
1) Containment Strategies
Containment limits damage while preserving evidence:
Containment Goals:
1. Stop ongoing damage
- Prevent lateral movement
- Stop data exfiltration
- Block C2 communication
2. Preserve evidence
- Don't destroy forensic artifacts
- Maintain chain of custody
- Document all actions
3. Maintain operations (if possible)
- Business continuity considerations
- Minimize disruption
- Enable investigation
Containment is NOT:
- Complete remediation
- Permanent fix
- Full investigation
- Recovery
Containment Approaches:
Short-term Containment (immediate):
- Isolate affected systems
- Block malicious IPs/domains
- Disable compromised accounts
- Disconnect from network (extreme)
Long-term Containment (temporary state):
- Rebuild system on isolated network
- Apply emergency patches
- Increase monitoring
- Implement temporary controls
Containment Decision Factors:
- Business criticality of affected systems
- Current attacker activity level
- Evidence preservation needs
- Available backup/recovery options
- Regulatory/legal requirements
Network Containment Options:
Least Disruptive → Most Disruptive:
1. Block specific IOCs
- Firewall rules for malicious IPs
- DNS sinkhole for C2 domains
- Minimal business impact
2. Segment affected systems
- VLAN isolation
- ACL restrictions
- Limited connectivity maintained
3. Disable network port
- System offline from network
- Local access still possible
- Moderate disruption
4. Physical disconnection
- Complete network isolation
- Maximum evidence preservation
- Highest business impact
5. Power off (last resort)
- Stops all activity
- May lose volatile evidence
- RAM contents lost
Endpoint Containment:
EDR-based containment:
- Network isolation via agent
- System still accessible for investigation
- Process blocking/killing
- File quarantine
Account containment:
- Disable user account
- Reset credentials
- Revoke sessions/tokens
- Increase authentication requirements
Application containment:
- Block malicious processes
- Disable compromised services
- Application whitelisting enforcement
Key insight: Start with least disruptive containment that achieves the goal. Escalate to more aggressive measures only if needed.
2) Evidence Preservation
Containment must preserve evidence for investigation and potential legal proceedings:
Order of Volatility:
Most volatile (collect first):
1. CPU registers, cache
2. RAM contents
3. Network connections
4. Running processes
5. Open files
Less volatile:
6. Disk contents
7. Logs on disk
8. Configuration files
9. Archived data
10. Backups
Why it matters:
- Volatile data disappears when system changes
- RAM lost on reboot
- Network connections close
- Processes terminate
- Collect volatile data before containment actions
Memory Acquisition:
# Capture RAM before other actions
Tools:
- WinPMEM (Windows)
- LiME (Linux Memory Extractor)
- Belkasoft RAM Capturer
- F-Response (enterprise)
Windows example (WinPMEM):
winpmem_mini_x64.exe memdump.raw
Linux example (LiME):
insmod lime.ko "path=/tmp/mem.lime format=lime"
Memory contains:
- Running malware
- Decrypted data
- Network connections
- Credentials in memory
- Process injection evidence
- Encryption keys
Disk Imaging:
# Create forensic image of disk
Tools:
- FTK Imager (free)
- dc3dd (Linux)
- Guymager
- EnCase
Best practices:
- Use write blocker if possible
- Create bit-for-bit copy
- Hash original and image (MD5/SHA256)
- Verify hashes match
- Document chain of custody
When full imaging isn't possible:
- Triage collection (key artifacts only)
- Targeted acquisition
- Logical copies of critical files
- Log exports
Chain of Custody:
Chain of Custody Documentation:
For each evidence item:
- Description of item
- Date/time collected
- Who collected it
- Where it was collected
- Hash values
- Storage location
- Everyone who accessed it
Transfer log:
| Date/Time | From | To | Purpose | Signature |
|-----------|------|-----|---------|-----------|
| 01/15 09:00 | JSmith | Evidence Locker | Storage | JS |
| 01/16 14:00 | Evidence | KJones | Analysis | KJ |
Why it matters:
- Legal admissibility
- Proof of integrity
- Investigation credibility
Key insight: Evidence that can't be proven intact is worthless in court. Chain of custody discipline matters from minute one.
3) Eradication
After containment, remove the threat completely:
Eradication Goals:
1. Remove all attacker access
- Malware removed
- Backdoors closed
- Compromised credentials reset
- Persistence mechanisms eliminated
2. Close the vulnerability
- Patch exploited systems
- Fix misconfigurations
- Address root cause
3. Verify clean state
- No remaining malware
- No active C2
- No suspicious processes
Eradication Techniques:
Malware Removal:
- Antivirus/EDR remediation
- Manual file deletion
- Registry cleanup
- Service removal
- Scheduled task removal
Account Remediation:
- Password resets (all compromised accounts)
- Credential rotation (service accounts)
- Token revocation
- MFA enforcement/reset
- Review account permissions
System Remediation:
- Patch vulnerable software
- Update configurations
- Remove unauthorized software
- Restore from known-good backup
Network Remediation:
- Update firewall rules
- Block attacker infrastructure
- Revoke VPN certificates
- Update DNS records
Rebuild vs. Clean:
Clean the existing system:
+ Faster return to service
+ Preserves configurations
- May miss hidden malware
- Rootkits can survive
- Less confidence in clean state
Rebuild from scratch:
+ High confidence in clean state
+ Eliminates all persistence
+ Opportunity to improve config
- Takes longer
- May lose configurations
- Requires good backups
When to rebuild:
- Rootkit suspected
- Kernel-level compromise
- Can't verify clean state
- Critical system worth the time
- Compliance requires it
When cleaning may suffice:
- Well-understood malware
- Limited, verified scope
- Non-critical system
- Time pressure extreme
Key insight: If you're not confident eradication was complete, the attacker will return. When in doubt, rebuild.
4) Recovery
Return to normal operations safely:
Recovery Phases:
1. Restore systems
- Rebuild or restore from backup
- Apply all patches
- Harden configurations
- Verify clean state
2. Validate functionality
- Test critical functions
- Verify integrations
- Check data integrity
- Confirm user access
3. Return to production
- Staged reconnection
- Increased monitoring
- User communication
- Support readiness
4. Monitor for recurrence
- Enhanced logging
- Additional alerts
- Threat hunting
- Watch for attacker return
Recovery Priorities:
Prioritize by:
Business criticality:
1. Systems that generate revenue
2. Systems that serve customers
3. Systems that support operations
4. Internal productivity systems
Dependencies:
- Infrastructure first (AD, DNS, network)
- Then dependent applications
- Then end-user systems
Example recovery order:
1. Active Directory (everything depends on it)
2. DNS and DHCP
3. Email (communication critical)
4. Core business applications
5. File servers
6. Workstations
Staged Recovery:
Don't restore everything at once:
Stage 1: Isolated testing
- Restore to isolated network
- Verify clean and functional
- Test security controls
Stage 2: Limited production
- Connect to production with monitoring
- Small user group tests
- Watch for issues
Stage 3: Full production
- Restore normal access
- Continue enhanced monitoring
- Ready to re-isolate if needed
Recovery validation checklist:
□ System boots and runs normally
□ All patches applied
□ Security software running
□ Logs flowing to SIEM
□ No suspicious processes
□ No unexpected network connections
□ User access verified
□ Data integrity confirmed
Key insight: Rushing recovery invites reinfection. Take time to verify clean state and implement improvements.
5) Communication During Incidents
Effective communication is critical throughout:
Internal Communication:
Status Updates:
- Regular cadence (hourly during active incident)
- Consistent format
- Clear current state
- Next steps identified
Update template:
INCIDENT: [Name/ID]
STATUS: [Active/Contained/Resolved]
TIME: [Current time UTC]
CURRENT SITUATION:
[2-3 sentence summary]
ACTIONS SINCE LAST UPDATE:
- [Action 1]
- [Action 2]
NEXT STEPS:
- [Planned action 1]
- [Planned action 2]
NEXT UPDATE: [Time]
Stakeholder Communication:
Different audiences need different information:
Technical Team:
- Detailed technical findings
- IOCs and TTPs
- Specific remediation steps
- Tool outputs
Management:
- Business impact
- Timeline and milestones
- Resource needs
- Risk assessment
Legal/Compliance:
- Regulatory implications
- Evidence preservation
- Notification requirements
- Liability concerns
Executives:
- High-level summary
- Business impact
- Recovery timeline
- Decisions needed
External (if needed):
- Customers
- Regulators
- Law enforcement
- Media
War Room Operations:
For major incidents, establish command center:
Physical or virtual:
- Dedicated space/channel
- All responders accessible
- Shared screens/documents
- Communication tools ready
War room discipline:
- IC controls the room
- One conversation at a time
- Document everything
- Regular breaks (fatigue kills effectiveness)
- Clear check-in/check-out
Information sharing:
- Central timeline document
- Shared evidence repository
- IOC list maintained
- Action tracker
Shift planning:
- 8-12 hour shifts maximum
- Overlap for handoff
- Written handoff notes
- No single point of failure
Key insight: Poor communication extends incidents. Keep everyone informed with the information they need in a format they understand.
Real-World Context: Incident Handling in Practice
Real incidents are messy and unpredictable:
Ransomware Response: Containment is urgent—every minute more systems get encrypted. Network segmentation and endpoint isolation happen simultaneously. Evidence preservation competes with containment speed. Recovery may take weeks.
Data Breach: Scope determination is critical. What data was accessed? Legal and regulatory notifications may be required within hours or days. Evidence preservation is paramount for potential litigation.
Active Intrusion: The attacker is watching. Containment actions may tip them off. Coordinated response— hitting all their access simultaneously—may be necessary.
MITRE ATT&CK Application:
- Identify Techniques Used: What TTPs did the attacker use?
- Anticipate Next Steps: What might they do next?
- Comprehensive Eradication: Address all techniques observed
Key insight: Every incident is different. Frameworks provide structure, but judgment adapts to circumstances.
Guided Lab: Containment Decision Exercise
Practice making containment decisions under pressure.