Shell + Markdown
Objectives
- Introduction to the Shell
- Shell Programming
- Introduction to Markdown
Topics
- Shell is central:
- Common interface to all computing machinery
- Folders, paths (absolute and relative), wildcards, globbing
- Permissions for files and directories
- Job control: piping, sequences, one output is another input
- Shell commands
- Regular Expressions (and
grep) - Shell programming
awkandsed- Intro to
diffandpatch - Markdown is a very powerful and popular "mark-up" language for text
- We will learn about markup, separating content from style, and see examples
- Key idea here: Markup is independent of rendered output
- Excellent topic once again for self-study and experimentation
- pandoc used as universal converter to/from
- RMarkdown convenience wrapper around pandoc calls
- e.g. to Javascript/HTML for (dynamic / web) presentations
- e.g. to pdf for printable writeups, presentations, books, ...
- A first look at the newer Quarto extensions to Markdown
Core Material
Lecture Slides
- Lecture 3 (2025): Shell I
- Lecture 4 (2025): Shell II
- Lecture 5 (2025): sed + awk
- Lecture 6 (2024): Markdown (and RMarkdown example file)
Lecture Videos
The core videos which complement the slides are the (captioned) versions of the 2021 course. As the captions may obstruct some of the displayed code, we also show the uncaptioned original versions. We complement these core content videos with current 'by the week' videos.
- Video 3 (2025): Week 2 (Box), also Video 3 (2025, captioned) (ClassTranscribe)
- Video 4 (2021): Shell I (Box, captioned), also Video 4 (2021): Shell I (uncaptioned)
- Video 5 (2021): Shell II (Box, captioned), also Video 5 (2021): Shell II (uncaptioned)
- Video 6 (2025): Week 3 (Box), also Video 6 (2025, captioned) (ClassTranscribe)
- Video 7 (2021): sed + awk (Box, captioned)
- Video 8 (2021): Markdown Intro (Box, captioned), also Video 8 (2021) (Box) (and RMarkdown example file)
- Video 9 (2021): Markdown Outputs (Box, captioned), also Video 9 (2021) (Box) (and RMarkdown example file)
Extras
- Small examples/ directory on box.com and as a downloadable tar.gz and zip archive
- Example data archive github repository and downloadable tar.gz and zip archive
Supplementary Material (not required, but helpful)
Extra Reading and Resources
- Shell scripting tutorial free site, pdf for sale
- Julia Evans 'zine' (not free but fairly inexpensive)
- Surprisingly difficult to find a good reading list:
- Unix for the Impatient an old favourite of mine; might be a good 2nd-hand pick-up
- Resources:
- Examples of tutorial-style sites
- Some historical context: In the beginning was the command-line by Stephenson.
- More advanced shell scripting resource: Pure Bash Bible
- Andy Oram on Power and Promise of awk
- Intro tutorial / overview: Understanding awk
- And news from August 2022: "Unix legend, who owes us nothing, keeps fixing foundational AWK code"
- Shell script checker can help to debug your shell script
- Welcome to the Linux command line for you and me offers friendly documentation
- The Art of the Commmand-Line is a useful (if dense) reference to "master the command line, in one page"
- RStudio RMarkdown cheatsheet (2.0) and reference guide
Julia Evans
A few of her comics made it onto the lecture slides. Here are some more related to these lectures:
- what's a shell
- why I love
bash bashvariablesbashbuiltinsbashifstatementsbashforloopsbashtricks- more
bashtricks bashfunctionsbashbrackets cheat sheet- shell history
bashparameter expansion- reading input in
bash bashpipesbashglobsbasherrorsshellcheckhead&tailsort&uniqfindgrepcat& friendssedawkxargs(for command-line jobs in pipes)- opening a file
- unix permissions
- processes
kill(interacts with processes)top(shows active processes)- pipes
- subshells
- environment variables