Study sheet: System Security Fundamentals

📋 Course Outline

1. Security Objectives and Properties
2. Control Hijacking and Buffer Overflows
3. Integer Overflow and Memory Safety
4. Browser Defenses and Isolation
5. Robust Software Development
6. Program Analysis Methods
7. Privileges and Access Control
8. Operating System Protection
9. Exploitation Techniques and Defenses
10. Fuzzing and Vulnerability Detection
11. Vulnerability Exploit and Attack

📖 1. Security Objectives and Properties

🔑 Key Concepts & Definitions

• System Security : the protection of computer systems, operating systems, applications, networks, and data from unauthorized access, modification, destruction, data theft, malware, vulnerability exploitation, and denial of service

📝 Essential Points

• The CIA triad consists of:
  • confidentiality
  • integrity
  • availability

📌 Authentication verifies who a user is, authorization determines what an authenticated user may do, and accountability tracks actions performed by users.

💡 Memory Hook

CIA: Confidentiality, Integrity, Availability

📖 2. Control Hijacking and Buffer Overflows

🔑 Key Concepts & Definitions

• Control Hijacking : an attack that manipulates a program's execution flow so that it executes unintended instructions or attacker-controlled code
• Buffer Overflow : a condition in which a program writes more data into a fixed-size memory buffer than the buffer can hold

★ Must-know

• If a buffer has a capacity of 10 bytes and receives 20 bytes, the extra data may overwrite adjacent memory.

• A stack-based buffer overflow can overwrite nearby stack data, including a return address, so that function return transfers execution to an unintended location.

Further detail

• Common causes include:
  • lack of bounds checking
  • unsafe string operations
  • incorrect memory management
  • improper input validation
  • programming errors

💡 Memory Hook

Unsafe memory write → corrupted control data → unintended execution

📖 3. Integer Overflow and Memory Safety

🔑 Key Concepts & Definitions

• Integer Overflow : a condition in which an arithmetic operation produces a value outside the range representable by the integer data type

★ Must-know

• An unsigned 8-bit integer represents values from 0 to 255, so 255 plus 1 wraps around to 0.

• An integer overflow can produce an unexpectedly small size, cause insufficient memory allocation, and lead to a potential buffer overflow.

Further detail

• The representable range of a 32-bit signed integer is $-2^{31}$ to $2^{31}-1$.

💡 Memory Hook

Arithmetic overflow → incorrect size → insufficient allocation → memory corruption

📖 4. Browser Defenses and Isolation

🔑 Key Concepts & Definitions

• Sandboxing : a security mechanism that executes a program inside a restricted environment with limited access to files, networks, hardware, processes, system calls, and sensitive data
• Isolation : the separation of processes, users, applications, or resources so that compromising one component does not automatically compromise the others

★ Must-know

📌 Sandboxing limits damage from compromised software but does not guarantee security because a sandbox escape vulnerability can let an attacker break outside the restricted environment.

Further detail

• Browser protections include:
  • sandboxing
  • ASLR
  • DEP/NX
  • site isolation
  • process isolation
  • the Same-Origin Policy

💡 Memory Hook

Sandboxing restricts actions, whereas isolation separates components

📖 5. Robust Software Development

🔑 Key Concepts & Definitions

• Robust Software : software that continues to behave safely and correctly when it receives unexpected input or encounters abnormal conditions

★ Must-know

• Input validation should check:
  • type
  • length
  • range
  • format
  • encoding

📌 The principle of least privilege requires each user, process, or application to have only the minimum privileges needed to perform its task.

Further detail

• Secure software practices include:
  • safe error handling
  • secure defaults
  • code review
  • unit testing
  • integration testing
  • security testing
  • fuzz testing
  • penetration testing

💡 Memory Hook

Validate → encode → handle errors → minimize privileges → test

📖 6. Program Analysis Methods

★ Must-know

📌 Static analysis examines software without executing it, dynamic analysis examines it during execution, and concolic analysis combines concrete execution with symbolic reasoning.

• Dynamic analysis can detect memory violations, crashes, invalid memory accesses, and resource problems, but it only explores executed paths.

• Concolic execution runs concrete inputs while maintaining symbolic constraints, derives path constraints, and generates new inputs to explore alternative paths.

Further detail

• Static analysis can detect:

  • possible buffer overflows
  • uninitialized variables
  • dead code
  • memory problems
  • dangerous function usage
  • injection vulnerabilities
  • incorrect data flows

• The main limitation of concolic analysis is path explosion, in which the number of execution paths grows extremely quickly.

💡 Memory Hook

SDC: Static, Dynamic, Concolic

📖 7. Privileges and Access Control

🔑 Key Concepts & Definitions

• Privilege : permission granted to a user, process, or program to perform a particular operation
• Access Control : the determination of which subjects may access which resources and which operations they are allowed to perform

📝 Essential Points

📌 Vertical privilege escalation moves from a low-privileged account to a high-privileged account, whereas horizontal privilege escalation moves to another account with a similar privilege level.

📌 Authentication answers who a user is, whereas authorization answers what that authenticated user is allowed to do.

• DAC lets resource owners determine access, MAC applies centrally enforced policies and labels, RBAC assigns permissions to roles, and ABAC bases decisions on attributes such as user, role, device, location, resource, time, and environment.

💡 Memory Hook

Authentication asks who you are, whereas authorization asks what you may do

📖 8. Operating System Protection

★ Must-know

• Operating-system security protects:
  • processes
  • memory
  • files
  • users
  • devices
  • system calls
  • the kernel
  • network resources

📌 Process isolation gives each process a protected address space so that one process should not directly access another process's memory.

• Memory pages may have read, write, and execute permissions, represented by R, W, and X; an R-X page is readable and executable but not writable.

Further detail

• Major operating-system security mechanisms include authentication, authorization, access control, process isolation, memory protection, privilege separation, secure boot, file permissions, auditing, logging, and security updates.

📖 9. Exploitation Techniques and Defenses

🔑 Key Concepts & Definitions

• Exploitation : the process of taking advantage of a software or system vulnerability to cause unintended behavior
• Code Injection : Code injection occurs when an attacker causes unintended code or commands to be interpreted or executed, as in SQL injection, command injection, or script injection.
• Use-After-Free : a vulnerability in which a program uses memory after that memory has already been released
• Return-Oriented Programming : an exploitation technique that uses existing instruction sequences called gadgets to produce desired behavior without injecting an entire new program

📝 Essential Points

• Important defenses include:
  • ASLR
  • DEP/NX
  • stack canaries
  • CFI
  • sandboxing
  • least privilege

💡 Memory Hook

Vulnerability → analysis → trigger → unintended behavior → impact

📖 10. Fuzzing and Vulnerability Detection

🔑 Key Concepts & Definitions

• Fuzzing : an automated testing technique that supplies large amounts of unexpected, malformed, random, or specially generated input to a program and monitors its behavior
• Coverage-Guided Fuzzing : fuzzing that monitors which program regions are executed and mutates inputs to explore new execution paths

📝 Essential Points

• A fuzzing campaign generates test input, runs the application, monitors for crashes or interesting behavior, saves useful inputs, and analyzes the results.

• Fuzzing types include:

  • mutation-based fuzzing
  • generation-based fuzzing
  • black-box fuzzing
  • white-box fuzzing
  • grey-box fuzzing

💡 Memory Hook

Generate → execute → monitor → save → analyze

📖 11. Vulnerability Exploit and Attack

📝 Essential Points

📌 A vulnerability is a weakness in a system, an exploit is a technique, code, or input that takes advantage of the weakness, and an attack is broader malicious activity using vulnerabilities or exploits to achieve a goal.

• The relationship among these concepts is vulnerability, exploit, attack, and impact.

💡 Memory Hook

Weakness → exploit → attack → impact

📊 Synthesis Tables

Program Analysis Comparison

Feature	Static	Dynamic	Concolic
Program execution	No	Yes	Yes
Actual input	Not necessarily	Yes	Yes
Symbolic constraints	Usually no	Usually no	Yes
Main limitation	False positives	Missed paths	Path explosion

Access Control Models

Model	Decision basis	Typical role
DAC	Resource-owner choice	Owner grants access
MAC	Central policies and labels	Highly controlled environments
RBAC	Assigned roles	Permissions follow roles
ABAC	User, device, location, time, and other attributes	Context-dependent access