# Lecture 2 – Command-line-Environment

### Overview

Lecture 1 introduced the shell as a launcher and compositor. This lecture goes deeper — into how shell programs **communicate**, **cooperate**, and **coexist** within a shared environment.

You'll learn the conventions every well-behaved CLI program follows: how it receives arguments, how it communicates status back to the shell, how it responds to system-level signals, and how to connect to remote machines seamlessly. You'll also learn how to make your terminal feel truly personal through dotfiles, aliases, and tmux.

This lecture is about mastering the **ecosystem** around shell programs, not just the programs themselves.

#### Key Takeaways

* Shell programs communicate through five channels: **arguments, streams, environment variables, return codes, and signals**.
* **Globs** (`*`, `?`, `{}`) are expanded by the shell *before* the program ever runs — programs never see them.
* **Pipelines are concurrent** — all programs in a pipeline start simultaneously; they don't wait for each other to finish.
* **stderr is not piped** — it flows to the terminal unless you explicitly redirect it.
* Exit codes of `0` mean success; any nonzero means failure. `&&` and `||` chain commands based on this.
* **Signals** like `SIGINT` (Ctrl-C) and `SIGTERM` are the OS's mechanism for telling processes what to do.
* **SSH key-based auth** is more secure and more convenient than passwords — always prefer it.
* **tmux** lets you persist terminal sessions, split panes, and keep work alive even after disconnecting.
* **Dotfiles** are the configuration layer of your entire development environment — version-control them.
* **Aliases and shell functions** eliminate repetitive typing and encode best-practice flags permanently.

***

### Core Concepts

#### The Five Communication Channels of a Shell Program

Every CLI program communicates with the world through five well-defined channels. Understanding all five is what separates shell beginners from power users.

| Channel                           | Direction         | Example                    |
| --------------------------------- | ----------------- | -------------------------- |
| **Arguments**                     | Caller → Program  | `ls -la /home`             |
| **Streams** (stdin/stdout/stderr) | Both              | `cat file \| grep pattern` |
| **Environment variables**         | Caller → Program  | `TZ=Asia/Tokyo date`       |
| **Return codes**                  | Program → Caller  | `$?`, `&&`, `\|\|`         |
| **Signals**                       | OS/User → Program | `Ctrl-C` sends `SIGINT`    |

Unlike a function in Python or Java where inputs and outputs are explicitly typed and declared, shell programs communicate through these **implicit, convention-based channels**. The shell itself never enforces them — it's purely social contract between tools.

***

#### Arguments and Flags

When you run `ls -la /home`, the shell sends the program a list of string arguments: `['-la', '/home']`. The program interprets them however it wants, but there's a strong convention:

* **Single-dash flags**: short, single-character (`-l`, `-a`, `-h`)
* **Double-dash flags**: long, descriptive (`--all`, `--human-readable`, `--help`)
* **Grouping**: `-l -a` and `-la` are equivalent for most programs
* **Order**: flag order usually doesn't matter

These aren't enforced by the shell — they're a **community convention** that nearly all tools follow. Violating them (e.g., using `+myoption`) would deeply confuse any developer familiar with Unix tools.

Inside a script, arguments are accessed with special variables:

| Variable      | Meaning                    |
| ------------- | -------------------------- |
| `$0`          | Name of the script/program |
| `$1`, `$2`, … | First, second, … argument  |
| `$@`          | All arguments as a list    |
| `$#`          | Number of arguments        |

```bash
#!/usr/bin/env bash
echo "Script name: $0"
echo "First arg:   $1"
echo "All args:    $@"
echo "Arg count:   $#"
```

***

#### Globs: Shell Pattern Expansion

Globs are patterns the **shell expands before the program runs**. The program never sees `*.py` — it receives `main.py utils.py helpers.py`. This is a critical mental model shift.

| Glob      | Matches                       |
| --------- | ----------------------------- |
| `*`       | Zero or more of any character |
| `?`       | Exactly one character         |
| `{a,b,c}` | Exactly `a`, `b`, or `c`      |
| `**`      | Recursive paths (zsh/bash 4+) |

```bash
# Delete all Python files in current directory
rm *.py

# Match single character: file1.txt, file2.txt, but NOT file10.txt
ls file?.txt

# Create multiple files at once
touch src/{main,utils,tests}.py
# Expands to: touch src/main.py src/utils.py src/tests.py

# Copy multiple script types to new directory
cp /project/{setup,build,deploy}.sh /newpath/

# Combine globs
mv *{.py,.sh} scripts/
# Moves all .py and .sh files into scripts/
```

> **Key insight:** If no files match a glob, bash raises an error by default (`no matches found`). This is why checking globs before destructive operations like `rm` is smart.

***

#### Streams: stdin, stdout, stderr

Every process has three open file descriptors at birth:

| Stream     | FD | Default destination | Purpose              |
| ---------- | -- | ------------------- | -------------------- |
| **stdin**  | 0  | Keyboard            | Input to the program |
| **stdout** | 1  | Terminal            | Normal output        |
| **stderr** | 2  | Terminal            | Errors and warnings  |

**The critical fact about pipelines:** Only stdout is piped. stderr continues flowing to the terminal even inside a pipeline. This is intentional — error messages from intermediate pipeline steps should be visible, not silently swallowed.

```bash
# stderr is NOT piped — error still appears on terminal
ls /nonexistent | grep "pattern"
# → ls: cannot access '/nonexistent': No such file or directory

# Redirect stderr to a file
ls /nonexistent 2> errors.txt

# Redirect both stdout and stderr to the same file
ls /nonexistent &> all_output.txt

# Discard stderr completely (silence errors)
some_command 2>/dev/null

# Discard ALL output
some_command > /dev/null 2>&1
```

**Pipelines are concurrent — not sequential.** When you run:

```bash
cat myfile | grep pattern | uniq -c
```

All three programs start at the same time. `cat` feeds data into the pipe as it reads; `grep` processes it as it arrives; `uniq` processes that in turn. This is not a three-step sequential process — it's three concurrent processes connected by pipes.

***

#### Environment Variables

Environment variables are key-value pairs that every process inherits from its parent. They're a **passive configuration channel** — programs can read them without needing explicit arguments.

```bash
# Assign a local shell variable (no spaces around =!)
foo=bar
echo "$foo"      # prints: bar
echo '$foo'      # prints: $foo  (single quotes = no interpolation)

# Assign for one command only (child inherits, but parent shell unchanged)
TZ=Asia/Tokyo date
echo $TZ         # empty — TZ was only set for 'date'

# Export to make it available to all subsequent child processes
export DEBUG=1
bash -c 'echo $DEBUG'   # prints: 1

# View all current environment variables
printenv

# Delete a variable
unset DEBUG
```

**Convention:** Environment variable names are `ALL_CAPS`. Local script variables are `lowercase`. This is convention only, not enforcement.

**Command substitution** — capture stdout of a command into a variable:

```bash
files=$(ls)
echo "$files" | grep README
```

**Process substitution** — use a command's output as a temporary file:

```bash
# Compare directory listings without creating temp files
diff <(ls src) <(ls docs)
```

`<(CMD)` runs `CMD`, writes its output to a temp file, and substitutes the filename. This is powerful when a program requires a filename argument but you want to feed it dynamic data.

***

#### Return Codes

Every program exits with a numeric code. By convention:

* `0` → success
* Non-zero → something went wrong (the specific number often encodes the error type)

```bash
# Check the return code of the last command
ls /existing-dir
echo $?    # 0

ls /nonexistent
echo $?    # 2 (non-zero = failure)
```

The shell's `&&` and `||` operators are **short-circuit evaluators** based on return codes:

```bash
# Run second command only if first succeeds (exit code 0)
mkdir /tmp/work && cd /tmp/work

# Run second command only if first fails (non-zero exit code)
ping -c1 server.example.com || echo "Server is unreachable"

# Chain: setup, build, deploy — stop at first failure
./configure && make && make install

# Retry fallback
use_fast_method || use_slow_fallback
```

`true` and `false` are programs that always succeed (exit 0) and always fail (exit 1) respectively — useful in scripts and testing.

***

#### Signals

Signals are **asynchronous software interrupts** delivered by the OS or terminal to a running process. The process can catch them, ignore them, or let the default action happen.

| Signal    | Key combo | Default action        | Meaning                     |
| --------- | --------- | --------------------- | --------------------------- |
| `SIGINT`  | `Ctrl-C`  | Terminate             | Interrupt from keyboard     |
| `SIGQUIT` | `Ctrl-\`  | Terminate + core dump | Quit from keyboard          |
| `SIGTSTP` | `Ctrl-Z`  | Pause (stop)          | Terminal stop               |
| `SIGTERM` | —         | Terminate gracefully  | Polite kill request         |
| `SIGKILL` | —         | Terminate immediately | Cannot be caught or ignored |
| `SIGHUP`  | —         | Terminate             | Terminal closed             |
| `SIGSTOP` | —         | Pause                 | Cannot be caught or ignored |

**Job control** — managing running processes:

```bash
# Start a long-running job
sleep 1000

# Pause it (sends SIGTSTP)
Ctrl-Z
# → [1]  + suspended  sleep 1000

# Resume in background
bg

# Resume in foreground
fg

# List all jobs in current session
jobs

# Start a command directly in background
sleep 2000 &

# Kill a specific job by number
kill %1

# Kill by signal type
kill -SIGTERM %2
kill -SIGKILL 12345   # by PID
```

**Preventing death on terminal close:**

```bash
# nohup: immune to SIGHUP when terminal closes
nohup long-running-script.sh &

# disown: detach already-running job from terminal
long-running-script.sh &
disown %1
```

**Trap: cleanup on exit (in scripts):**

```bash
#!/usr/bin/env bash
cleanup() {
    echo "Cleaning up..."
    rm -f /tmp/mytemp.*
}
trap cleanup EXIT         # always runs on exit
trap cleanup SIGINT SIGTERM   # also on Ctrl-C or kill
```

***

#### Remote Machines with SSH

SSH (Secure Shell) is the standard tool for connecting to remote machines. It provides an encrypted, authenticated shell session over a network.

```bash
# Connect to a remote server
ssh alice@server.example.com

# Run a single command remotely (non-interactive)
ssh alice@server ls /var/log

# Run a pipeline — ls runs remotely, wc runs locally
ssh alice@server ls | wc -l

# Run an entire pipeline on the remote server
ssh alice@server 'ls | wc -l'
```

**Key-based authentication** — always prefer over passwords:

```bash
# Generate a new key pair (Ed25519 is modern and secure)
ssh-keygen -a 100 -t ed25519 -f ~/.ssh/id_ed25519

# Copy your public key to the remote server
ssh-copy-id -i ~/.ssh/id_ed25519 alice@remote

# Manual equivalent
cat ~/.ssh/id_ed25519.pub | ssh alice@remote 'cat >> ~/.ssh/authorized_keys'
```

**SSH config** — define named hosts so you never type long commands again:

```
# ~/.ssh/config

Host myserver
    User alice
    HostName 172.16.174.141
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host *.company.com
    User alice
    IdentityFile ~/.ssh/id_ed25519
```

After this config, `ssh myserver` is all you need.

**File transfer:**

```bash
# Copy local file to remote
scp report.pdf alice@server:/home/alice/

# Copy remote file to local
scp alice@server:/var/log/app.log ./

# Sync directories (only transfers changes)
rsync -avz ./project/ alice@server:~/project/

# Resume an interrupted transfer
rsync --partial -avz ./bigfile.tar.gz alice@server:~/
```

***

#### Terminal Multiplexers (tmux)

tmux solves two problems at once:

1. **Multiple sessions in one terminal** — split into panes, open new windows, switch between them
2. **Session persistence** — detach from a session and reattach later, even after disconnecting from SSH

**tmux hierarchy:** Session → Windows → Panes

All tmux commands start with the **prefix key**: `Ctrl-b` (press, release, then press the next key).

**Sessions:**

| Command            | Action                          |
| ------------------ | ------------------------------- |
| `tmux`             | Start new unnamed session       |
| `tmux new -s work` | Start session named "work"      |
| `tmux ls`          | List all sessions               |
| `tmux a`           | Attach to last session          |
| `tmux a -t work`   | Attach to session named "work"  |
| `Ctrl-b d`         | **Detach** from current session |

**Windows (tabs):**

| Keybinding | Action                |
| ---------- | --------------------- |
| `Ctrl-b c` | Create new window     |
| `Ctrl-b N` | Go to window number N |
| `Ctrl-b n` | Next window           |
| `Ctrl-b p` | Previous window       |
| `Ctrl-b ,` | Rename current window |
| `Ctrl-b w` | List all windows      |

**Panes (splits):**

| Keybinding     | Action                                           |
| -------------- | ------------------------------------------------ |
| `Ctrl-b "`     | Split horizontally (top/bottom)                  |
| `Ctrl-b %`     | Split vertically (left/right)                    |
| `Ctrl-b ←↑→↓`  | Move between panes                               |
| `Ctrl-b z`     | Zoom/unzoom current pane                         |
| `Ctrl-b [`     | Enter scroll mode (arrow keys scroll, `q` exits) |
| `Ctrl-b space` | Cycle pane layouts                               |

***

#### Dotfiles and Shell Customization

**Dotfiles** are configuration files with names starting with `.` (hidden by default in `ls`). They live in your home directory and control the behavior of nearly every developer tool.

| Tool | Dotfile                        |
| ---- | ------------------------------ |
| bash | `~/.bashrc`, `~/.bash_profile` |
| zsh  | `~/.zshrc`                     |
| git  | `~/.gitconfig`                 |
| vim  | `~/.vimrc`                     |
| ssh  | `~/.ssh/config`                |
| tmux | `~/.tmux.conf`                 |

**The golden rule of dotfiles:** Keep them in a single version-controlled folder and symlink them into place.

```bash
# Typical dotfiles repo setup
mkdir ~/dotfiles
cd ~/dotfiles
git init

# Move configs here and symlink back
mv ~/.bashrc ~/dotfiles/bashrc
ln -s ~/dotfiles/bashrc ~/.bashrc
```

Benefits:

* **Portability** — same environment on every machine in minutes
* **History** — see exactly what you changed and when
* **Recovery** — never lose your setup to a disk wipe
* **Sharing** — publish on GitHub, learn from others' configs

***

#### Aliases

Aliases are short-form substitutions that the shell expands before running any command.

```bash
# Syntax (no spaces around =)
alias alias_name="command arg1 arg2"

# Common useful aliases
alias ll="ls -lh"
alias la="ls -A"
alias lla="la -l"

# Safety nets
alias mv="mv -i"        # prompt before overwriting
alias rm="rm -i"        # prompt before deleting
alias cp="cp -i"        # prompt before overwriting

# Git shortcuts
alias gs="git status"
alias gc="git commit"
alias gp="git push"
alias gl="git log --oneline --graph"

# Fix common typos
alias sl=ls
alias dc=cd

# Navigate up quickly
alias ..="cd .."
alias ...="cd ../.."

# Check an alias definition
alias ll
# → ll='ls -lh'

# Run original command bypassing alias
\ls

# Remove an alias
unalias la
```

Aliases are defined in your `~/.bashrc` (or `~/.zshrc`). They're loaded each time you open a new shell. Add them there to make them permanent.

***

#### AI in the Shell

AI tools can now integrate directly into your shell workflow:

```bash
# Generate shell commands from English descriptions
llm cmd "find all python files modified in the last week"
# → find . -name "*.py" -mtime -7

# Process unstructured text through a pipeline
INSTRUCTIONS="Extract just the username from each line, one per line, nothing else"
llm "$INSTRUCTIONS" < users.txt
```

Tools like **Claude Code** act as a meta-shell — accepting natural language and translating it into multi-step shell operations, file edits, and more.

***

### Mental Model

#### The Shell Is a Coordination Layer

Think of the shell not as a program runner, but as a **process coordinator**. When you type a pipeline, the shell:

1. Forks all processes simultaneously
2. Connects their stdin/stdout with in-memory pipe buffers
3. Steps aside and lets them run

The shell's job is done before any program outputs a single character. This is why `Ctrl-C` in a pipeline kills all processes — the shell sends `SIGINT` to the entire process group.

```
Your terminal
      │
      ▼
   Shell ──── forks ────────────────────────────────────┐
      │                                                  │
   [cat] ──stdout──► [pipe] ──stdin──► [grep] ──stdout──► [sort] ──► terminal
      │                                   │
   (runs immediately)               (runs immediately)
```

All three programs are alive and running the moment you press Enter.

***

#### Environment Variables: The Implicit Configuration Layer

Think of environment variables as the **ambient context** a process lives in. When your shell spawns a child process, it hands it a copy of its entire environment.

```
Shell environment: { HOME, PATH, USER, TERM, ... }
         │
         │  fork()
         ▼
  Child process gets a COPY of environment
  (changes in child do NOT affect parent)
```

This is why `export` matters. Without `export`, a variable lives only in your current shell and dies when that shell exits. With `export`, it's copied into every child process.

***

#### Return Codes as Boolean Logic

Return codes map perfectly onto boolean logic:

| Return code | Boolean value | Meaning |
| ----------- | ------------- | ------- |
| `0`         | `true`        | Success |
| non-zero    | `false`       | Failure |

This is why `&&` and `||` work the way they do:

```
cmd1 && cmd2
  → "run cmd1; if it succeeded (returned true), run cmd2"

cmd1 || cmd2
  → "run cmd1; if it failed (returned false), run cmd2"
```

This maps directly to short-circuit boolean evaluation in programming languages — `true && X` evaluates `X`; `false || X` evaluates `X`.

***

#### Signals: Asynchronous Control Messages

Signals are the OS's way of tapping a process on the shoulder mid-execution. Unlike streams (which a process reads when it wants), signals are **delivered immediately**, interrupting whatever the process is doing.

A process has three choices when it receives a signal:

1. **Handle it** — run a custom signal handler function
2. **Ignore it** — do nothing (not possible for `SIGKILL` or `SIGSTOP`)
3. **Default action** — usually terminate or pause

The key practical lesson: `SIGKILL` (signal 9) **cannot be caught or ignored**. It's the OS-level nuclear option — always terminates immediately, which can leave orphaned child processes or corrupt state. Always try `SIGTERM` first.

***

### Commands and Syntax

#### Command: `printenv`

```bash
printenv
```

Prints all environment variables in the current shell session.

```bash
printenv                # print all variables
printenv PATH           # print a specific variable
printenv | grep JAVA    # find Java-related variables
```

***

#### Command: `export`

```bash
export VARIABLE=value
```

Sets a variable and marks it for export to all child processes.

```bash
export DEBUG=1
export PATH="$PATH:/usr/local/myapp/bin"
export EDITOR=vim
```

***

#### Command: `alias`

```bash
alias name="command"
```

Creates a command shortcut. Permanent when added to `~/.bashrc`.

```bash
alias ll="ls -lh"
alias gs="git status"
alias ..="cd .."
alias ll          # view an alias definition
unalias ll        # remove an alias
```

***

#### Command: `kill`

```bash
kill [signal] PID_or_job
```

Sends a signal to a process.

```bash
kill %1              # send SIGTERM to job 1
kill -SIGTERM 12345  # send SIGTERM to PID 12345
kill -SIGKILL 12345  # force-kill (last resort)
kill -0 12345        # check if process exists (no signal sent)
kill -l              # list all signals
```

***

#### Command: `jobs`

```bash
jobs
```

Lists all background and suspended jobs in the current terminal session.

```bash
jobs          # list jobs
jobs -l       # include PIDs
```

***

#### Command: `fg` / `bg`

```bash
fg [%job_number]
bg [%job_number]
```

Bring a job to the foreground or send it to the background.

```bash
fg        # resume the most recent paused job in foreground
fg %2     # resume job 2 in foreground
bg %1     # resume job 1 in background
```

***

#### Command: `nohup`

```bash
nohup command &
```

Runs a command immune to `SIGHUP` (terminal close). Output goes to `nohup.out`.

```bash
nohup ./long-analysis.sh &
nohup python3 server.py > server.log 2>&1 &
```

***

#### Command: `pgrep` / `pkill`

```bash
pgrep pattern
pkill pattern
```

Find or kill processes by name pattern — no need to look up PIDs manually.

```bash
pgrep sleep        # find PIDs of all 'sleep' processes
pkill sleep        # kill all 'sleep' processes
pkill -SIGTERM python  # gracefully kill all python processes
pgrep -af python   # show full command line of matching processes
```

***

#### Command: `ssh`

```bash
ssh [user@]hostname [command]
```

Connects to a remote machine or runs a command on it.

```bash
ssh alice@192.168.1.10            # interactive session
ssh alice@server 'df -h'          # run single command
ssh alice@server 'ls /var' | wc   # mix local and remote
ssh -p 2222 alice@server          # custom port
```

***

#### Command: `ssh-keygen`

```bash
ssh-keygen -a 100 -t ed25519 -f ~/.ssh/id_ed25519
```

Generates a new SSH key pair. Use Ed25519 (modern, fast, secure).

| Flag          | Meaning                                                 |
| ------------- | ------------------------------------------------------- |
| `-t ed25519`  | Key type (Ed25519 recommended)                          |
| `-f filename` | Output file                                             |
| `-a 100`      | Key derivation iterations (higher = slower brute-force) |

***

#### Command: `ssh-copy-id`

```bash
ssh-copy-id -i ~/.ssh/id_ed25519 user@remote
```

Copies your public key to a remote server's `authorized_keys`. After this, you can log in without a password.

***

#### Command: `scp`

```bash
scp source destination
```

Securely copies files over SSH.

```bash
scp file.txt alice@server:/tmp/         # local → remote
scp alice@server:/tmp/log.txt ./        # remote → local
scp -r ./project alice@server:~/        # copy directory
```

***

#### Command: `rsync`

```bash
rsync [flags] source destination
```

Syncs files efficiently — only transfers what changed.

```bash
rsync -avz ./src/ alice@server:~/src/   # sync local to remote
rsync -avz --delete ./src/ server:~/src/ # mirror (delete remote extras)
rsync --partial -avz bigfile.tar.gz server:~/  # resume interrupted transfer
```

| Flag        | Meaning                                                |
| ----------- | ------------------------------------------------------ |
| `-a`        | Archive mode (preserves permissions, timestamps, etc.) |
| `-v`        | Verbose                                                |
| `-z`        | Compress during transfer                               |
| `--partial` | Keep partial transfers (enables resume)                |
| `--delete`  | Delete files at destination that don't exist at source |

***

#### Command: `tmux`

```bash
tmux              # start new session
tmux new -s name  # start named session
tmux ls           # list sessions
tmux a -t name    # attach to named session
```

See the full keybinding reference in the Core Concepts section above.

***

#### Command: `trap`

```bash
trap 'command' SIGNAL
```

Register a handler to run when a signal is received. Essential for script cleanup.

```bash
trap 'rm -f /tmp/lockfile' EXIT
trap 'echo "Interrupted!"; exit 1' SIGINT
```

***

### Command Flow Diagrams

#### Pipeline Concurrency Model

```mermaid
sequenceDiagram
    participant Shell
    participant cat
    participant grep
    participant uniq

    Shell->>cat: fork() + connect stdout→pipe1
    Shell->>grep: fork() + connect stdin←pipe1, stdout→pipe2
    Shell->>uniq: fork() + connect stdin←pipe2

    Note over cat,uniq: All three processes start simultaneously

    cat-->>grep: streams data via pipe1
    grep-->>uniq: streams filtered data via pipe2
    uniq-->>Shell: writes final output to terminal
```

***

#### Environment Variable Inheritance

```mermaid
graph TD
    SH["Parent Shell\nHOME=/home/alice\nPATH=...\nDEBUG=1 (exported)"]
    C1["Child Process 1\n(inherits all exported vars)\nDEBUG=1"]
    C2["Child Process 2\n(inherits all exported vars)\nDEBUG=1"]
    C3["Grandchild Process\n(inherits from Child 1)\nDEBUG=1"]

    SH -->|fork + exec| C1
    SH -->|fork + exec| C2
    C1 -->|fork + exec| C3

    style SH fill:#2d4a22,color:#fff
    style C1 fill:#1a3a5c,color:#fff
    style C2 fill:#1a3a5c,color:#fff
    style C3 fill:#4a1a5c,color:#fff
```

***

#### Signal Delivery Flow

```mermaid
graph LR
    User["User presses Ctrl-C"] --> Terminal["Terminal"]
    Terminal --> Shell["Shell detects SIGINT"]
    Shell --> PG["Sends SIGINT to\nentire process group"]
    PG --> P1["Process 1\n(terminates)"]
    PG --> P2["Process 2\n(terminates)"]
    PG --> P3["Process 3\n(custom handler?)"]
    P3 -->|"if handler installed"| H["Custom handler runs\n(cleanup, log, ignore)"]
```

***

#### SSH Authentication Flow

```mermaid
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: SSH connection request
    Server-->>Client: Send random challenge
    Client->>Client: Sign challenge with private key
    Client->>Server: Send signed challenge
    Server->>Server: Verify signature with stored public key
    Server-->>Client: Access granted ✅

    Note over Client,Server: Private key never leaves the client machine
```

***

#### tmux Architecture

```mermaid
graph TB
    S1["Session: work"]
    S2["Session: personal"]

    W1["Window 1: editor"]
    W2["Window 2: server"]
    W3["Window 3: logs"]

    P1["Pane: vim"]
    P2["Pane: file tree"]
    P3["Pane: node server"]
    P4["Pane: tail -f"]

    S1 --> W1
    S1 --> W2
    S1 --> W3
    W1 --> P1
    W1 --> P2
    W2 --> P3
    W3 --> P4
    S2 -.->|detached, survives SSH disconnect| S2

    style S1 fill:#2d4a22,color:#fff
    style S2 fill:#4a2d22,color:#fff
```

***

### Command Pipeline Examples

#### Example 1: Find Your Most-Used Commands

```bash
history | awk '{$1=""; print substr($0,2)}' | sort | uniq -c | sort -rn | head -10
```

| Step | Command                             | What it does                              |
| ---- | ----------------------------------- | ----------------------------------------- |
| 1    | `history`                           | Outputs command history with line numbers |
| 2    | `awk '{$1=""; print substr($0,2)}'` | Strips the line number prefix             |
| 3    | `sort`                              | Sorts all commands alphabetically         |
| 4    | `uniq -c`                           | Counts consecutive duplicate lines        |
| 5    | `sort -rn`                          | Re-sorts by count, highest first          |
| 6    | `head -10`                          | Keeps only the top 10                     |

***

#### Example 2: Run a Command Until It Fails, Capture Output

```bash
#!/usr/bin/env bash
count=0
while true; do
    count=$((count + 1))
    output=$(./flaky-script.sh 2>&1)   # capture stdout AND stderr
    if [[ $? -ne 0 ]]; then
        echo "Failed after $count runs"
        echo "Output was:"
        echo "$output"
        break
    fi
done
```

This pattern is invaluable for debugging intermittent failures.

***

#### Example 3: Process Substitution for Side-by-Side Diffs

```bash
diff <(ls src/) <(ls docs/)
```

Without process substitution, you'd need temp files:

```bash
ls src/ > /tmp/src_list.txt
ls docs/ > /tmp/docs_list.txt
diff /tmp/src_list.txt /tmp/docs_list.txt
rm /tmp/src_list.txt /tmp/docs_list.txt
```

Process substitution does all of that in a single, clean expression.

***

#### Example 4: Safe Pipeline with Error Checking

```bash
# Run the pipeline, capture exit status of each stage
set -o pipefail    # make pipeline fail if ANY stage fails

grep "ERROR" app.log | sort | uniq -c | sort -rn > error_report.txt
if [[ $? -ne 0 ]]; then
    echo "Pipeline failed — no ERROR lines found or file missing"
fi
```

> By default, a pipeline's exit code is the exit code of the **last** command only. `set -o pipefail` changes this so that any failure in the pipeline causes the whole pipeline to report failure.

***

#### Example 5: Background Job with Clean Output

```bash
# Run in background, redirect output to avoid polluting terminal
./build-system.sh > /tmp/build.log 2>&1 &
BUILD_PID=$!

echo "Build started (PID $BUILD_PID)"
echo "Tail log: tail -f /tmp/build.log"

# Wait for it and check result
wait $BUILD_PID
if [[ $? -eq 0 ]]; then
    echo "Build succeeded!"
else
    echo "Build failed — check /tmp/build.log"
fi
```

***

### Real World Workflows

#### Set Up SSH Config for a Project Server

```bash
# Add to ~/.ssh/config
cat >> ~/.ssh/config << 'EOF'
Host devserver
    User deploy
    HostName 10.0.1.50
    IdentityFile ~/.ssh/id_ed25519
    Port 22
EOF

# Now connect with just:
ssh devserver

# Copy files easily
scp build.tar.gz devserver:/var/www/

# Sync project
rsync -avz --exclude='node_modules' ./project/ devserver:~/project/
```

***

#### Start a tmux Session for a Long-Running Dev Environment

```bash
# Create a named session with three windows
tmux new -s myproject

# Inside tmux:
# Window 1: editor
# Ctrl-b , → rename to "editor"

# Ctrl-b c → new window
# Window 2: server → npm start
# Ctrl-b , → rename to "server"

# Ctrl-b c → new window  
# Window 3: Ctrl-b " to split, run tests in one pane and logs in another

# Detach and walk away: Ctrl-b d
# Come back any time: tmux a -t myproject
```

***

#### Create a Dotfiles Bootstrap Script

```bash
#!/usr/bin/env bash
# install.sh — run on a fresh machine to set up your environment

DOTFILES="$HOME/dotfiles"

# Clone your dotfiles repo
git clone git@github.com:yourname/dotfiles.git "$DOTFILES"

# Symlink each config file
ln -sf "$DOTFILES/bashrc"    "$HOME/.bashrc"
ln -sf "$DOTFILES/vimrc"     "$HOME/.vimrc"
ln -sf "$DOTFILES/tmux.conf" "$HOME/.tmux.conf"
ln -sf "$DOTFILES/gitconfig" "$HOME/.gitconfig"

mkdir -p "$HOME/.ssh"
ln -sf "$DOTFILES/ssh_config" "$HOME/.ssh/config"

echo "Dotfiles installed. Reload your shell: source ~/.bashrc"
```

***

#### Debug a Flaky Background Service

```bash
# Check if it's running
pgrep -af myservice

# Watch it in real time
tail -f /var/log/myservice.log

# Kill it gracefully
pkill -SIGTERM myservice

# Wait for it to die, then restart
sleep 2 && pgrep myservice || ./start-service.sh
```

***

### Productivity Tricks

#### Reverse History Search with fzf

Standard `Ctrl-R` searches sequentially. With `fzf` integration, it becomes an interactive fuzzy-finder over your entire history:

```bash
# Install fzf (macOS)
brew install fzf
$(brew --prefix)/opt/fzf/install

# Now Ctrl-R shows an interactive, filterable history panel
```

***

#### One-Time Environment Variables

You can set an environment variable for a single command without affecting your current shell:

```bash
TZ=America/New_York date     # print New York time, without changing your TZ
DEBUG=1 ./myapp              # run with debug mode, just this once
NODE_ENV=production npm start
```

***

#### The `--` Separator

`--` tells a program "stop parsing flags, everything after this is a positional argument". Essential when filenames start with `-`:

```bash
touch -- -myfile       # create a file literally named "-myfile"
rm -- -myfile          # delete it (rm -myfile would be interpreted as a flag)
grep -- -v file.txt    # search for the literal string "-v"
```

***

#### Combining Aliases into Chains

```bash
# In ~/.bashrc
alias update="sudo apt update && sudo apt upgrade -y && sudo apt autoremove -y"
alias dev="cd ~/projects && tmux new -s dev"
alias deploy="./build.sh && rsync -avz dist/ server:~/www/"
```

***

#### Quick File Transfer via SSH

```bash
# Pipe a command's output directly to a remote file
tar czf - ./project | ssh server 'cat > ~/project-backup.tar.gz'

# Pull a remote file and extract locally
ssh server 'tar czf - /var/log' | tar xzf -
```

***

### Common Mistakes

#### ❌ Space Around `=` in Variable Assignment

**Wrong:**

```bash
foo = bar    # Shell tries to run program named "foo" with args "=" and "bar"
```

**Correct:**

```bash
foo=bar
```

**Why:** The shell uses spaces to split tokens into arguments. `foo = bar` is parsed as: run program `foo`, pass it `=` and `bar` as arguments.

***

#### ❌ Forgetting to Quote Variables Containing Spaces

**Wrong:**

```bash
files=$(ls my\ docs/)
grep "README" $files    # word-splits the filenames!
```

**Correct:**

```bash
grep "README" "$files"
```

**Why:** An unquoted variable undergoes **word splitting** — spaces inside it become argument separators. Always quote: `"$variable"`.

***

#### ❌ Assuming Pipelines Are Sequential

**Wrong mental model:**

```
# "cat finishes, then grep starts, then sort starts"
cat file | grep pattern | sort
```

**Reality:** All three start simultaneously. `cat` streams data as it reads; `grep` processes it in real time. For large files, this matters enormously for performance.

***

#### ❌ Using `SIGKILL` First

**Wrong:**

```bash
kill -9 12345    # immediately and violently
```

**Correct:**

```bash
kill -SIGTERM 12345   # politely ask to exit
sleep 3
kill -0 12345 && kill -SIGKILL 12345   # only force-kill if still alive
```

**Why:** `SIGKILL` cannot be caught, so the process has no chance to clean up (close files, release locks, flush buffers). Always try `SIGTERM` first.

***

#### ❌ Backgrounding Without Redirecting Output

**Wrong:**

```bash
./noisy-script.sh &
# Script's stdout now pollutes your terminal while you type
```

**Correct:**

```bash
./noisy-script.sh > /tmp/output.log 2>&1 &
# Or use tee to see it and save it
./noisy-script.sh 2>&1 | tee /tmp/output.log &
```

***

#### ❌ `curl | bash` Without Reviewing the Script

**Wrong:**

```bash
curl https://example.com/install.sh | bash   # you're running unreviewed code!
```

**Safer:**

```bash
curl -fsSL https://example.com/install.sh -o install.sh
less install.sh    # read it first
bash install.sh
```

***

#### ❌ Not Using a Passphrase on SSH Private Keys

Your private key is essentially a password. Without a passphrase, anyone who steals the key file can impersonate you on every server you have access to. Use `ssh-agent` to avoid typing the passphrase repeatedly:

```bash
# Generate with passphrase
ssh-keygen -t ed25519 -a 100

# Add to agent (type passphrase once per session)
ssh-add ~/.ssh/id_ed25519
```

***

### Exercises

#### Beginner Exercises

1. **Explore arguments:** Run `echo $0`, `echo $#`, `echo $@` in your terminal. What do they show? Now create a script `args.sh` that prints "Script: $0", "Count: $#", and "Args: $@", then run it with `bash args.sh foo bar baz`.
2. **Glob practice:** In `/tmp`, create files `file1.txt`, `file2.txt`, `file10.txt`, `note.md`. Now try:
   * `ls /tmp/file?.txt` — what matches?
   * `ls /tmp/file*.txt` — what matches? What's the difference?
   * `ls /tmp/{file1,note}*` — what expands to what?
3. **Redirect stderr:** Run `ls /nonexistent`. Now run `ls /nonexistent 2>/dev/null`. Explain what changed. Now run `ls /nonexistent 2>&1 | cat` — where does the error go?
4. **Return codes:** Run `true; echo $?` and `false; echo $?`. What do you see? Now try `ls /tmp && echo "success"` and `ls /nonexistent && echo "success"`. Why does one print and the other doesn't?
5. **Create a useful alias:** Add `alias ll="ls -lh --color=auto"` to your `~/.bashrc`. Reload it with `source ~/.bashrc` and try `ll`.

***

#### Intermediate Exercises

6. **marco and polo (from MIT):** Write a `~/.bashrc` function `marco` that saves your current directory, and `polo` that returns you to it:

   ```bash
   marco() {
       export MARCO_DIR="$(pwd)"
   }

   polo() {
       cd "$MARCO_DIR" || echo "No marco location saved"
   }
   ```

   Source your bashrc and test: `cd /tmp; marco; cd /var; polo` — did it return you to `/tmp`?
7. **Trap for cleanup:** Write a script that creates a temp file, sleeps for 30 seconds, and always cleans up the temp file even if you press `Ctrl-C`:

   ```bash
   #!/usr/bin/env bash
   tmpfile=$(mktemp /tmp/test.XXXXXX)
   trap "rm -f $tmpfile; echo 'Cleaned up.'" EXIT
   echo "Working with $tmpfile"
   sleep 30
   ```
8. **Process substitution diff:** Run `diff <(printenv | sort) <(export | sort)`. What's different about the output of `printenv` vs `export`? Why?
9. **Job control:** Run `sleep 10000` in your terminal. Press `Ctrl-Z` to pause it. Run `jobs`. Resume it in the background with `bg`. Use `pgrep sleep` to find its PID. Kill it with `pkill sleep`. Verify with `jobs` again.
10. **Top 10 commands:** Run:

    ```bash
    history | awk '{$1=""; print substr($0,2)}' | sort | uniq -c | sort -rn | head -10
    ```

    Create at least 3 aliases for the commands you use most.

***

#### Advanced Challenge

11. **`pidwait` function:** Write a bash function `pidwait` that takes a PID and waits (polling every 2 seconds) until the process with that PID no longer exists, then prints "Process finished":

    ```bash
    pidwait() {
        local pid=$1
        while kill -0 "$pid" 2>/dev/null; do
            sleep 2
        done
        echo "Process $pid has finished"
    }
    ```

    Test it:

    ```bash
    sleep 30 &
    PID=$!
    pidwait $PID
    # Should print "Process finished" ~30 seconds later
    ```
12. **Flaky script debugger:** Given this unreliable script that fails \~1% of the time:

    ```bash
    #!/usr/bin/env bash
    n=$(( RANDOM % 100 ))
    if [[ $n -eq 42 ]]; then
        echo "Something went wrong" >&2
        exit 1
    fi
    echo "Everything went according to plan"
    ```

    Write a wrapper script that runs it in a loop until it fails, captures stdout and stderr to separate files, and reports how many attempts it took.
13. **SSH + tmux workflow:** On a remote machine (or VM), start a tmux session named `dev`, split it into 3 panes (editor, server, logs), start a process in each pane, then detach. Reconnect from a second terminal tab and verify the session is intact. Kill the original terminal entirely — the tmux session should survive.
14. **Dotfiles setup:** Create a `~/dotfiles` directory, move your `.bashrc` (or `.zshrc`) there, create a symlink back with `ln -s`, and initialize a git repo. Add at least one alias, one export, and one function. Push to GitHub.

***

### Summary

| Topic            | Core Idea                                                                        |
| ---------------- | -------------------------------------------------------------------------------- |
| **Arguments**    | `$1`-`$9`, `$@`, `$#`, `$0` — how programs receive input                         |
| **Globs**        | `*`, `?`, `{}` expanded by shell *before* program runs                           |
| **Streams**      | stdin (0), stdout (1), stderr (2) — only stdout is piped by default              |
| **Pipelines**    | Concurrent — all programs start simultaneously                                   |
| **Env vars**     | Inherited by child processes; `export` to propagate                              |
| **Return codes** | 0 = success; non-zero = failure; `&&` / `\|\|` chain on them                     |
| **Signals**      | Async interrupts; `Ctrl-C` = SIGINT; `Ctrl-Z` = SIGTSTP; `kill -9` = last resort |
| **Job control**  | `&`, `fg`, `bg`, `jobs`, `kill`, `nohup`, `disown`                               |
| **SSH**          | Key auth over passwords; config file for shortcuts; rsync for transfers          |
| **tmux**         | Sessions → Windows → Panes; persist across disconnects                           |
| **Dotfiles**     | Version-controlled configs, symlinked from `~/dotfiles`                          |
| **Aliases**      | `alias name="cmd"` in `~/.bashrc`; `\cmd` to bypass                              |

#### Most Important Commands from This Lecture

```
export      – set and propagate environment variables
printenv    – inspect environment
alias       – create command shortcuts
kill        – send signals to processes
jobs        – list background/stopped jobs
fg / bg     – bring jobs fore/back-ground
nohup       – protect from terminal close
pgrep/pkill – find/kill by name
ssh         – connect to remote machines
ssh-keygen  – generate key pairs
ssh-copy-id – install public key on server
scp / rsync – transfer files securely
tmux        – terminal multiplexer (sessions, windows, panes)
trap        – register cleanup handlers for signals
```

#### What's Next

In **Lecture 3 – Development Environment and Tools**, you'll learn to edit files entirely from the keyboard using the most powerful text editor in the Unix ecosystem — one that's available on virtually every server you'll ever SSH into.

***

*Source:* [*MIT Missing Semester – Command-line Environment*](https://missing.csail.mit.edu/2026/command-line-environment/) *Licensed under* [*CC BY-NC-SA 4.0*](https://creativecommons.org/licenses/by-nc-sa/4.0)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://shankar-lab.gitbook.io/mylearning/the-missing-semester-of-your-cs-education/lecture-2-command-line-environment.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.