• Welcome to CS 3650!
• What is this course about?
• This class is an introduction to low-level programming, operating system concepts and implementation, and how to interact with the services that an OS provides
• It is a class about abstraction: A modern computer is a complex hardware system, built on very simple principles (1, 0, logic gates). An OS manages this complexity and provides abstractions for working with a computer system
• It is also a class about resource management
• Compared to other classes: we go to more detail than CS5007 – which is an Align masters course, but we cover only part of what 5600 covers and in less detail
• Completing 3650 should give you a good basis for tackling a more advanced course, such as 5600

Topics

• Roughly speaking, this course has several topics:
  1. Computer Systems Fundamentals - tools and languages
     • Terminal, C, Assembly, compiler toolchain
  2. CPU Virtualization
     • Processes
  3. Computer Architecture
     • Memory/Cache/Virtual Memory
  4. Concurrency
     • Threads/Locks/Semaphores
     • Parallelism
  5. Persistence
     • Storage Devices
     • File Systems
  6. Other Selected Topics Throughout The Semester
     • Debugging
     • Instrumentation
     • (Testing)
• Most of this course will talk about a special program called the operating system
• With that in mind, one of our goals is demystifying computer systems – both hardware and systems software
• There is no magic – just 0s and 1s and several thousands/millions lines of code that make things work the way we are used to nowadays
• One thing to realize is that things started relatively simply – the big operating systems with tons of features and fancy user interfaces are a consequence of evolution of software engineering techniques and consumer expectations
• The other thing to realize is that systems we use were designed and implemented by people like you!
• In a lot of ways computer science is both a science and an art – we base our code on solid mathematical principles, as well as physics, but a lot of creativity has gone into getting us where we are now

Meta

• The first thing you should do, is to familiarize yourself with the course resources
• We have a website
• It contains the Syllabus: read it today, then again before next class and prepare questions if something isn’t clear. It is important to familiarize yourself with the course policies regarding evaluation, lateness, academic integrity, etc.
• We also have a Canvas page and we will use it mostly to publish assignments and quizzes. We may also use it for section-specific announcements, such as class cancellations, to avoid spamming the whole class
• Course communication will primarily happen via Piazza, please make sure to follow the link from the website
• Programming assignments and projects will be submitted via Gradescope
• We will also use Github to distribute starter code and lab assignments

Evaluation

Assignments and Projects

• Most of your grade will come from assignments
• There will be about 8 assignments and 2 projects
• Out of the 8 assignments, the first 4 are strictly individual, the remainder + the projects can be completed in pairs
• You can partner across sections

Projects

• There will be two “projects”
• These are longer (2 weeks), more substantial programming exercises that will require you to plan and/or experiment more
• As such, the description will be more vague than with assignments – you are expected to do more reading, thinking, and asking

Labs

• We provide “labs” as a means to practice implementation techniques and tools
• The idea is to provide exercises related to the week’s topic, which will prepare you better for tackling that or next week’s assignment/project
• These will be graded mostly on effort – the intention is to encourage you to do the exercises as preparation for assignments
• Ideally, we would like to provide you some class time (30-60 mintes) every week to work on the labs, but if we need more time to cover topics, the “lab” will be purely a take-home exercise

Quizzes

• Almost weekly, there will be a quiz on the topics from class
• The intention is to make you engage with the material
• Questions will be from lectures and readings

Course Materials/Equipment

• A laptop
• The laptop’s operating system shouldn’t matter, however, having a Unix-like environment (Linux, macOS, WSL, *BSD, …) is an advantage
• We will provide you with a cloud-based Virtual Machine for you to work on
• We will use Linux for most of the course
• With any programming assignment, the assumption is, that you have tested your code on the provided VM. Regrade requests based on “it worked on my machine” will be rejected

Texts

Main texts

• OSTEP (most)
• Dive into Systems (parts)

Auxiliary

• Low-Level Programming
• The C Programming Language
• Lecture notes/slides
• Other linked resources

• While we don’t expect you to read each provided reading word by word, we expect you to skim it and be aware of where to find relevant information
• Most assignments and projects assume you’ve done the reading

Communications

• Questions regarding course content should be directed to Piazza
• Do not post snippets of your own code publicly (see cheating)
• Try to formulate your question to be about a concept, rather than a specific part of your code
• If you do want to ask a question about your code and you want to share a snippet, make a private Piazza post and address it to all instructors
• Do not post private questions addressed to a single individual

Email

• If you are sending an email, make sure you include the class code in the subject, e.g., [CS3650] Meeting request
• If relevant, include which section you are in
• If you do not get a reply, please send a reminder – emails sometimes get lost under a heap
  • Including the class in the email subject generally helps

Expectations and Teaching Style

• Everyone learns differently
• This class: mix of lectures, exercises, written resources (textbooks, articles, manuals, documentation, source code, …)
• (Ferd) I generally avoid slides and prefer live collaborative coding and using the white-/blackboard for figures, writing notes in a text editor, etc.
• Systems programming is based on a large body of knowledge that we cannot possibly cover in this course. You will need (learn to) to look for and use resources and do some research on your own
• Our goal is to help you become an independent learner and developer – this is an important skill in any worthwhile coding job
• Part of success (not only in this class) is figuring out what works for you! We can guide you through that process, provide resources and hints, but ultimately it is up to you
• This is a very hands-on class – We will build things!
• Allow yourself to make mistakes – this means starting early and experimenting
• Do not be afraid to be wrong
• Do readings (see above)
• Do ask questions
• Come to office hours
• Our expectation is that you know at least one programming language reasonably well – for most of you, this will be probably Java
• You can expect to get some exposure to x86-64 assembly and get a good base in C

The next steps

Operating Systems

• What operating system(s) do you use?
• What is an operating system, anyway?
• An OS is any and all software that sits between a user program (including the libraries it uses) and the hardware – basically a layer that sits between user software and the hardware
• A resource manager and allocator
  • Decides between conflicting requests for hardware access
  • Attempts to be efficient and fair
• OS is a control program
  • Controls execution of user programs
  • Prevents errors and improper use

+---------------+
|     User      |
+-------+-------+
        |
+-------+-------+
| Applications  |
+-------+-------+
        |
+-------+-------+
| Operating Sys |
+-------+-------+
        |
+-------+-------+
|   Hardware    |
+---------------+

• There are two common OS families nowadays:
  1. POSIX (Unix-like)
     • Linux, BSDs, macOS, Android, iOS, QNX
  2. Windows
• There are other operating systems, usually specialized to a specific domain
• We will work in a POSIX environment, specifically Linux
• How do you interact with an OS?

What is Linux?

• Linux is a family of free open source operating systems
  • That means the code is freely available, and you can contribute to the project!
• Created by Linus Torvalds as a small hobby project (see email quote below)
• Variants (distributions): Ubuntu, Debian, Fedora, Gentoo Linux, Arch Linux, CentOS,
• They all operate under roughly the same core code, which is called the kernel.
• Often they differ by the software, user interface, and configuration settings.
• So very often Linux software for one flavor of Linux will run on the other with few or no changes.
• Generally we (as systems programmers) like Linux, because it is a clean and hackable operating system.
• However, it has a large code base
• When many folks think of Unix-like operating systems, they may think of a hacker using a ‘command-line interface’ to program.

Linux Announcement

This is the message Linus sent to the comp.os.minix Usenet group (something like a mailing list) after about 2 months of work on his new OS:

From: torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds)
Newsgroups: comp.os.minix
Subject: What would you like to see most in minix?
Summary: small poll for my new operating system
Message-ID:
Date: 25 Aug 91 20:57:08 GMT
Organization: University of Helsinki


Hello everybody out there using minix -

I'm doing a (free) operating system (just a hobby, won't be big and
professional like gnu) for 386(486) AT clones.  This has been brewing
since april, and is starting to get ready.  I'd like any feedback on
things people like/dislike in minix, as my OS resembles it somewhat
(same physical layout of the file-system (due to practical reasons)
among other things).

I've currently ported bash(1.08) and gcc(1.40), and things seem to work.
This implies that I'll get something practical within a few months, and
I'd like to know what features most people would want.  Any suggestions
are welcome, but I won't promise I'll implement them :-)

              Linus (torvalds@kruuna.helsinki.fi)

PS.  Yes - it's free of any minix code, and it has a multi-threaded fs.
It is NOT protable (uses 386 task switching etc), and it probably never
will support anything other than AT-harddisks, as that's all I have :-(.

Interacting with an OS

• How do you interact with your OS of choice?
• You are probably using a graphical user interface of some sort
• Part of this course is to learn to use a command line interface
• The command line interface of Unix(-like) operating systems is traditionally called a shell
• We use a shell by using a terminal (to be more exact a terminal emulator application)
• Shell and terminal are commonly used interchangeably, but they are different programs (ask in a lecture if you want to know more :))
• If you use a macOS, Linux or another Unix-like OS, you most likely have a terminal application installed by default - look for something called “Terminal” or “xterm”
• If you are on Windows, the best is to use WSL2 with Windows Terminal (https://learn.microsoft.com/en-us/windows/wsl/install, https://medium.com/@awlucattini_60867/getting-started-with-wsl2-c11826654776) but many of the things we need you to do can also be done via PowerShell

Why a command line?? I love my windows and buttons!

• You may be asking yourself why we bother with a command line/shell
• The main reason is, that you can do a lot of things more efficiently from a shell - once you learn the basics
• It is easier to diagnose errors
• It is also easier to automate tasks and build complex tasks from relatively simple parts - this is very helpful for compiling and testing programs
• Pretty much all Unix shells support scripting – the command line program is actually an implementation of a Turing-complete language
• It is also easier to replicate a sequence of task on multiple machines (e.g., software installation and setup)
• It is easier and more seamless to work with remote machines (which we will be doing a lot in this course)
• Caveat: there is a slight learning curve!
• We will learn a little bit about the command line in the first lab this week

ssh

• A tool that we will expect you to use is ssh, aka, secure shell
• SSH is both a protocol and an implementation
• It allows us to run programs remotely on other machines
• By default, it runs a shell
• We will learn about ssh in the first lab
• You can try it out – for this, make sure you have a Khoury account
  1. Open a terminal
  2. Run ssh <your_khoury_username>@login.khoury.northeastern.edu, replacing <your_khoury_username> with your Khury account name
  3. Try some commands: ls -al, whoami, man ls
  4. Once you are done, type exit (or press Control-D)

Man pages

• Almost anything on Linux (or macOS) has a manual page
• The man system is very useful
• Try man ls to see the usage information about the ls command
• Or man less to read about the less command
• You can also man man

The C Compiler and Assembly

• Here’s our first C program, the traditional “Hello World”

  #include <stdio.h>
  
  int main(int argc, char **argv) {
    puts("Hello, World!");
    return 0;
  }

• Try to compile it on the login server (via ssh), or you can use a VM or you can try natively if you have a Linux installation

• Open a text editor (on most Linux distributions, you can use nano, emacs, vim – be warned, Vim has a steep learning curve: if you get stuck trying to exit press Esc and enter :wqa! and Enter)

• Enter the above code (type it, don’t copy) and save the file as hello.c

• You can compile this program using gcc (The GNU C Compiler), invoking it as follows:

  gcc hello.c -o hello

• This invocation asks GCC to compile the file hello.c and produce an executable named simply hello

• You can then run the program using ./hello (= “run the executable hello in the current directory”)