CS 6301--002: Systems Security and Binary Code Analysis Monday and Wednesday: 10:00AM-11:15AM ECSN 2.110 $Id: fall2013a.t2t, v1.0, last updated %%mtime(%c) Exp $ %! Target: %! Options: --toc --css-sugar --encoding=iso-8859-1 %! Style: tech.css %! PreProc: %! PostProc: --------------------------------------------------------------------------- === Course Overview === CS 6301 is a graduate level, research oriented, systems and software security class. The goal of this course is to understand the low-level system details with the real system implementations from compiler, linker, loader, to OS kernel and computer architectures, examine the weakest link in each system component, explore the left bits and bytes after all these transformations, and study the state-of-the-art offenses and defenses. The learning outcome is students shall be able to understand how an attack is launched (e.g., how an expoit is created), and how to do the defense (e.g., developing OS patches, analyzing the binary code, and detecting intrusions) In particular, we will cover - **Binary code reverse engineering**. Static binary code analysis. Dynamic Binary code instrumentation. Data flow analysis, and control flow analysis. Malware packing and unpacking. - **Memory exploits**. We will investigate the unsafe but widely used system programming language C, cover typical vulnerabilities such as buffer overflows, format strings, integer overflows, etc. How to create robust shell code using such as ROP, HeapSpray. - **OS Kernel Internals**. What's the behavior when a program is running on top of OS. Why we use paging. How virtual to physical address translation is performed. How MMU (e.g., TLB) helps this. How OS manage files, and disks. How can we model the program behavior when sitting at OS layer. We will use both Linux and Windows as working kernel. - **Linker and Loader Internals**. How a program can be dynamically linked, and what an attacker can do to cheat the system and meanwhile what we can do to protect the system. - **Kernel-level Defense**, how can we defend against the common exploits, techniques including such as hypervisor level virtual machine introspection, or kernel level ASLR, and DEP, NX-bits. - **User-level Defense**. Safe library, Compiler extension, Binary Transformation/Rewriting, Runtime Verification. This class will have heavy-hands on projects. The students will be able to get the experience on how to build real systems with virtual machine monitors, and process instrumentations. --------------------------------------------------------------------------- === Course Schedule === Please login your elearning website and download the lecture notes. || Week | Topic To Be Covered | 0 | Course Overview | || Understanding Binary Code Analysis || | 1 | Binary Code/Data Representation | | 2 | Program Representation | | 3 | Dynamic Binary Instrumentation (PIN, Valgrind, Qemu) | | 4 | Principles of Program Analysis | | 5 | Design and Implementation of a Data Flow Analysis (taint analysis) | || Understanding the OS Kernel || | 6 | Understanding the OS Architecture | | 7 | Process Management | | 8 | Virtual Memory | | 9 | File System | || Beyond OS Kernel || | 10 | Executable File Format: ELF, PE, Library Interposition | | 11 | Revealing Internals of Compiler (gcc), Linker, and Loader | | 12 | Virtual Machine Introspection (QEMU/VirtualBox/Xen/KVM) | || Other Techniques, Tools, and Applications || | 13 | Symbolic Execution and Whitebox Fuzzing | | 14 | Exploits: Buffer Overflows, Heap Overflow, Integer Overflow | | 15 | Robust Exploits: ROP shellcode, Heap Spray | | 16 | Fighting for Malware: Unpack, Disassemble, Decompile | --------------------------------------------------------------------------- === Office Hours === Wednesday, Friday 4PM - 6PM --------------------------------------------------------------------------- === Prerequisites === Solid programming/development skills (Assembly, C, C++, Unix) are required for this class. "Operating System", "Compilers", and "Computer Security", are the least prerequisites for this class. In particular, for UTD student - CS 3340 Computer Architecture - CS 3376 C/C++ Programming in a UNIX Environment - CS 4348 Operating Systems Concepts - CS 4393 Computer and Network Security - CS 4394 Implementation of Modern Operating Systems Note for undergraduate students who may be interested in taking this class, please be aware that the class is designed for graduate students, you are encouraged to attend the first lecture and then talk to the instructor. --------------------------------------------------------------------------- === Course Projects === - Dynamic Taint Analysis - Reverse Engineering - Virtual Machine Introspection - ... --------------------------------------------------------------------------- === Course Policy === ==== Grading Policy ==== - 10% Class participation - 20% Small assignments - 70% Course Projects - Exceptional work will be rewarded appropriately ==== Late Policy ==== No late submission. ==== Collaboration Policy ==== Students are encouraged to collaborate, particularly on the course project. But we will limit the team member to at most two students. ==== Cheating Policy ==== We will strictly follow the university policy on cheating and plagiarism which is available [here http://www.utdallas.edu/judicialaffairs/UTDJudicialAffairs-policies.html]. Please [avoid http://www.utdallas.edu/judicialaffairs/UTDJudicialAffairs-AvoidDishonesty.html]. There are also several examples of [Scholastic Dishonesty http://www.utdallas.edu/judicialaffairs/UTDJudicialAffairs-Basicexamples.html] If you have any questions regarding this issue, please contact the instructor. === Acknowledgememt/References === - [Understanding the Linux Kernel. http://oreilly.com/catalog/9780596005658] - [Penetration Testing and Vulnerability Analysis http://pentest.cryptocity.net/] - [Computer Systems: A Programmer's Perspective http://csapp.cs.cmu.edu/] - [Software Security at CMU http://www.ece.cmu.edu/~dbrumley/courses/18732-f11] - [Computer Security at UC Berkeley http://inst.eecs.berkeley.edu/~cs161/archives.html] - [Computer and Network Security at Stanford https://courseware.stanford.edu/pg/courses/lectures/170183] - [Computer and Network Security at MIT http://courses.csail.mit.edu/6.857/2011/handouts] - [Secure Programming at iSecLab http://www.iseclab.org/secprog/] %!include: ''sp12.js'' --------------------------------------------------------------------------- [HOME index.html] [SOURCE %%infile]