Reading Time: 9 minutes

In the part 1 of our blog we cover the in depth theoy of what is Process in Operating System. But that is not enough and not going to make you survive a prodcution crash. Lets do some hands on and under stand it more. Head over to your terminal and launch your shell (it is also a process). Type the below command.

ps #shows the processes that are running. 

ps commands stands for process state. The output will look like the following.

ubuntu:~$ ps
    PID TTY          TIME CMD
   1617 pts/1    00:00:00 bash
   1749 pts/1    00:00:00 bash
   1881 pts/1    00:00:00 ps
ubuntu:~$ 

Now lets break down each part to understand what is in it for us.

Column Headers:

• PID – Process ID (It is an unique identifier). Eevry process gets one.
• TTY – Terminal associated with the process. Not every process needs one!
• TIME – CPU time used by the process.
• CMD – Command name that started the process

Now we have the column headers lets understand the content in it. Based on the SS we have 3 processes.

The Processes:

1. Process 1617 (bash)
   • A bash shell running in pseudo-terminal pts/1
   • Used virtually no CPU time (00:00:00)
2. Process 1749 (bash)
   • Another bash shell in the same terminal pts/1
   • This suggests you started a new bash session from within the first one
   • Also used no measurable CPU time
3. Process 1881 (ps)
   • The ps command itself that you just ran
   • Shows up in its own output because it was running when it took the snapshot
   • Ran in the same terminal pts/1

What This Output Tells Us

The ps command shows itself because it captures running processes at that moment.

In the context of processes, TTY stands for “TeleTYpewriter” and refers to the terminal or console that a process is associated with. We have two bash shells running (one spawned from the other). All processes are in the same terminal session (pts/1) and all the processes are using minimal CPU time (typical for idle shells).

In our ps output, the TTY column shows which terminal each process is connected to:

• pts/1 – This stands for “pseudo-terminal slave” and indicates the processes are running in a pseudo-terminal. The number (1) identifies which specific pseudo-terminal session.
• Pseudo-terminals are created when you open terminal emulators, SSH sessions, or use terminal multiplexers like tmux/screen.

Here’s what this means for your processes:

• The bash shells (PIDs 1617 and 1749) are both attached to the same pseudo-terminal pts/1
• The ps command itself is also running in that same terminal

Other common TTY values you might see:

• tty1, tty2, etc. – Physical console terminals (like when you press Ctrl+Alt+F1)
• ? – Processes not associated with any terminal (typically daemons or background services)
• console – System console

The TTY association is important because:

1. It determines where the process sends its output
2. It defines which terminal receives the process’s input
3. It’s used for process group management and signal delivery
4. When you close a terminal, processes associated with that TTY typically receive a SIGHUP signal

You can see more detailed TTY information with ps -ef or ps aux which we will cover later.

How many processes are there?

We could only see only 3 processes running in the system. Is that all ? Nope! Not at all. Our system actually has hundreds or thousands of other processes running, but ps alone doesn’t show. But how do we see all the processes running in the system? specifically –

• Processes from other terminal sessions

• System processes (systemd, kernel threads, daemons)

• Other users’ processes

• Background processes without terminals (TTY = ?)

Lets see how to get all the process in the system.

ps -e #shows all the processes in the system

Here is the part of the output.

ubuntu:~$ ps -e
    PID TTY          TIME CMD
      1 ?        00:00:01 systemd
      2 ?        00:00:00 kthreadd
      3 ?        00:00:00 pool_workqueue_release
      4 ?        00:00:00 kworker/R-rcu_g
      5 ?        00:00:00 kworker/R-rcu_p
      6 ?        00:00:00 kworker/R-slub_
      7 ?        00:00:00 kworker/R-netns
      9 ?        00:00:00 kworker/0:0H-kblockd
     12 ?        00:00:00 kworker/R-mm_pe
     13 ?        00:00:00 rcu_tasks_kthread
     14 ?        00:00:00 rcu_tasks_rude_kthread
     15 ?        00:00:00 rcu_tasks_trace_kthread
     16 ?        00:00:00 ksoftirqd/0
     17 ?        00:00:00 rcu_preempt

Remeber, the TTY is shown as ? because these are system processes that are not associated with any terminal or console. Here’s why:

System Processes vs User Processes

System processes (like the ones you’re seeing) run in the background and don’t need terminal interaction:

• systemd (PID 1) – The init system that starts and manages other processes
• kthreadd (PID 2) – Kernel thread daemon that creates other kernel threads
• kworker processes – Kernel worker threads that handle various kernel tasks
• rcu_ processes* – Read-Copy-Update subsystem threads for kernel synchronization

These processes:

• Start at boot time before any terminals are available
• Run in kernel space or as system daemons
• Don’t need user interaction – they work entirely in the background
• Don’t produce output meant for users – they communicate through system logs instead

When you see TTY vs ?

• TTY (like pts/1, tty1) – Interactive processes that need terminal I/O
• ? (question mark) – Background processes, daemons, and kernel threads

But we can format it better and with more information with the ‘f’ parameter.

How to format the process output?

ps -ef  # format the output with more information. 

It will give you more information with the follwing formatted view.

ubuntu:~$ ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 05:39 ?        00:00:01 /sbin/init
root           2       0  0 05:39 ?        00:00:00 [kthreadd]
root           3       2  0 05:39 ?        00:00:00 [pool_workqueue_release]
root           4       2  0 05:39 ?        00:00:00 [kworker/R-rcu_g]
root        1189       1  0 05:39 ?        00:00:01 /opt/theia/node /opt/theia/browser-app/src-gen/backend/main.js /root --hostname=0.0.0.0 --port 4020
root        1226       1  0 05:39 ?        00:00:02 /bin/runtime-info-service
root        1237       1  0 05:39 ?        00:00:00 bash -c while true; do /bin/kc-terminal -p 40200 --writable -t disableLeaveAlert=true bash; done
root        1238    1237  0 05:39 ?        00:00:00 /bin/kc-terminal -p 40200 --writable -t disableLeaveAlert true bash
root        1570    1189  0 05:44 ?        00:00:04 /opt/theia/node /opt/theia/node_modules/@theia/core/lib/node/messaging/ipc-bootstrap --nsfwOptions=
root        1593    1189  0 05:44 pts/0    00:00:00 /bin/bash
root        1617    1238  0 05:44 pts/1    00:00:00 bash

We can dig into a specific process with formatted info like following. Most of the time you will target a specific process only. so remeber this command.

ps -fp process_id #shows the details of particular process.

ubuntu:~$ ps -fp 1593
UID          PID    PPID  C STIME TTY          TIME CMD
root        1593    1189  0 05:44 pts/0    00:00:00 /bin/bash
ubuntu:~$ 
ubuntu:~$ 

But we want to know more and more info. We can use this command.

ps -l #extend the info list (like process state) 

ubuntu:~$ ps -lf
F S UID          PID    PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
0 S root        1617    1238  0  80   0 -  1147 do_wai 05:44 pts/1    00:00:00 bash
0 S root        1749    1617  0  80   0 -  1147 do_wai 05:50 pts/1    00:00:00 bash
4 R root        2207    1749  0  80   0 -  1985 -      06:12 pts/1    00:00:00 ps -lf
ubuntu:~$ 

Complete Output Analysis

F (Flags) – Process State Flags

What: Octal representation of process flags
Why: The kernel needs to track special conditions about processes

All Possible Flag Values (octal):

• 0 = No flags set (normal process)
• 1 = Forked but didn’t exec
• 2 = Used super-user privileges
• 4 = Dumped core / being traced
• 8 = Called exit()
• 10 = Process is a kernel thread
• 20 = Process is being ptraced
• 40 = Process is stopped
• 100 = Process used VM86 mode

Your processes:

• 0 (bash processes) = Normal processes, no special flags
• 4 (ps command) = Being traced (because it’s examining other processes)

---

S (State) – Process Status

What: Single character showing what the process is currently doing
Why: Scheduler needs to know which processes can run vs which are waiting

All Possible Process States:

• D = Uninterruptible sleep (usually I/O)
• I = Idle kernel thread
• R = Running or runnable (on run queue)
• S = Interruptible sleep (waiting for event)
• T = Stopped by job control signal
• t = Stopped by debugger during tracing
• W = Paging (not valid since 2.6 kernel)
• X = Dead (should never be seen)
• Z = Zombie (terminated but not reaped by parent)

Additional state modifiers (can be combined):

• < = High-priority (not nice to other users)
• N = Low-priority (nice to other users)
• L = Has pages locked into memory
• s = Session leader
• l = Multi-threaded
• + = In foreground process group

Your processes:

• S (bash processes) = Sleeping, waiting for user input or events
• R (ps command) = Running/runnable, actively executing

---

UID (User ID)

What: Which user account owns this process
Why: Security – determines file access permissions and resource limits

Possible values: Any username or numeric UID

• System users: root, daemon, nobody, www-data
• Regular users: ubuntu, john, admin, etc.

Your processes: All show root = These processes have full system privileges

---

PID (Process ID)

What: Unique number assigned to each running process
Why: System needs a way to reference and manage individual processes

Possible range: 1 to 2²² (4,194,304) on most systems

• PID 1 is always init/systemd
• PIDs are recycled when processes die

Your processes: 1617, 1749, 2207 = Sequential assignment as processes were created

---

PPID (Parent Process ID)

What: PID of the process that created this one
Why: Process hierarchy – when parent dies, children need to be handled

Your process hierarchy:

• 1238 → 1617 → 1749 → 2207
• Shows chain: some parent → bash → bash → ps

---

C (CPU Usage Percentage)

What: Recent CPU usage (not current instant usage)
Why: Helps scheduler make decisions about which processes get CPU time

Possible range: 0-99 (percentage of CPU used recently)

Your processes: All 0 = These are lightweight processes using minimal CPU

---

PRI (Priority)

What: Scheduling priority (0-139, lower = higher priority)
Why: Determines which process gets CPU when multiple want to run

Complete priority ranges:

• 0-99 = Real-time priorities (kernel processes)
• 100-139 = Normal priorities (user processes)
• Higher number = lower priority

Your processes: All 80 = Default user process priority (120 – nice value of 0)

---

NI (Nice Value)

What: User-adjustable priority modifier (-20 to +19)
Why: Allows users to make processes “nicer” (lower priority) or more aggressive

Complete nice range:

• -20 = Highest user priority (least nice to others)
• 0 = Default priority
• +19 = Lowest user priority (most nice to others)

Your processes: All 0 = Default nice value, no priority adjustment

---

ADDR (Address)

What: Memory address where process image starts
Why: Historical – used for debugging, mostly obsolete now

Modern systems: Always shows - (not meaningful anymore)

Your processes: All - = Modern systems don’t display this meaningfully

---

SZ (Size) – Virtual Memory Pages

What: Virtual memory size in pages (usually 4KB per page)
Why: Memory management – kernel tracks how much virtual space each process uses

How pages work:

• Most systems use 4KB pages (4096 bytes per page)
• This is virtual memory, not physical RAM usage
• Includes code, data, libraries, stack, heap

Memory size ranges you might see:

• Small utilities: 500-2000 pages (2-8MB)
• Shell processes: 1000-3000 pages (4-12MB)
• Text editors: 2000-10000 pages (8-40MB)
• Web browsers: 50000+ pages (200MB+)
• Kernel threads: Often 0 (no userspace memory)

Your processes:

• 1147 (both bash processes): 1147 pages × 4KB = ~4.6MB virtual memory
  • Why this size: Bash needs memory for command history, variables, loaded libraries (libc, libreadline), stack space
• 1985 (ps command): 1985 pages × 4KB = ~7.9MB virtual memory
  • Why larger: ps must load additional libraries to read /proc filesystem, parse process information, format output

---

WCHAN (Wait Channel)

What: Kernel function name where process is sleeping
Why: Debugging – shows exactly what system call or condition process waits for

Common wait states:

• do_wait = Parent waiting for child process
• poll_schedule_timeout = Waiting for I/O with timeout
• tcp_recvmsg = Waiting for network data
• pipe_wait = Waiting for pipe I/O
• select = Waiting in select() system call
• futex_wait_queue_me = Waiting on futex (fast userspace mutex)
• schedule = Generic scheduler wait
• - = Not waiting (running)

Your processes:

• do_wai (bash processes) = Waiting for child processes (truncated display of “do_wait”)
• - (ps command) = Currently running, not waiting

---

STIME (Start Time)

What: When process began execution
Why: Tracking process lifetime and debugging long-running processes

Format: HH:MM for today, or Mon DD / MMM DD for older processes

Your processes: 05:44, 05:50, 06:12 = Chronological order showing process creation times

---

TTY (Terminal)

What: Which terminal/console the process is associated with
Why: Determines where input/output goes, signal delivery

All Terminal Types:

• console = System console
• tty[1-6] = Virtual consoles (Ctrl+Alt+F1-F6)
• pts/[n] = Pseudo-terminal (SSH, terminal emulator)
• ? = No controlling terminal (daemon processes)

Your processes: All pts/1 = Same pseudo-terminal session (likely SSH or terminal emulator)

---

TIME (CPU Time)

What: Total CPU time consumed since process started
Why: Resource accounting and performance monitoring

Format: HH:MM:SS showing cumulative CPU usage

Your processes: All 00:00:00 = Very brief execution times (less than 1 second total CPU time)

---

CMD (Command)

What: The actual command that started the process
Why: Identify what each process is doing

Your processes:

• bash = Bash shell processes (nested shells)
• ps -lf = The monitoring command you just executed

---

You would notice the most important PPID i.e. Parent Process ID. One process may generate another process. You can map this relation too with ‘–forest’ command. Observe the follwoing output.

The key insight is that both bash processes are <strong>sleeping</strong> (S state) waiting for input, while the ps command is <strong>running</strong> (R state).

ubuntu:~$ ps -ef --forest
UID          PID    PPID  C STIME TTY          TIME CMD
root           2       0  0 05:39 ?        00:00:00 [kthreadd]
root           3       2  0 05:39 ?        00:00:00  \_ [pool_workqueue_release]
root           4       2  0 05:39 ?        00:00:00  \_ [kworker/R-rcu_g]
root           5       2  0 05:39 ?        00:00:00  \_ [kworker/R-rcu_p]
root           6       2  0 05:39 ?        00:00:00  \_ [kworker/R-slub_]
root           7       2  0 05:39 ?        00:00:00  \_ [kworker/R-netns]
root           9       2  0 05:39 ?        00:00:00  \_ [kworker/0:0H-kblockd]
dhcpcd       828       1  0 05:39 ?        00:00:00 dhcpcd: enp1s0 [ip4] [ip6]
root         829     828  0 05:39 ?        00:00:00  \_ dhcpcd: [privileged proxy] enp1s0 [ip4] [ip6]
dhcpcd       868     829  0 05:39 ?        00:00:00  |   \_ dhcpcd: [BPF ARP] enp1s0 172.30.1.2
dhcpcd      1088     829  0 05:39 ?        00:00:00  |   \_ dhcpcd: [DHCP6 proxy] fe80::6143:e218:e882:bd0c
dhcpcd      1100     829  0 05:39 ?        00:00:00  |   \_ dhcpcd: [BOOTP proxy] 172.30.1.2
dhcpcd       830     828  0 05:39 ?        00:00:00  \_ dhcpcd: [network proxy] enp1s0 [ip4] [ip6]
dhcpcd       831     828  0 05:39 ?        00:00:00  \_ dhcpcd: [control proxy] enp1s0 [ip4] [ip6]
systemd+    1128       1  0 05:39 ?        00:00:00 /usr/lib/systemd/systemd-timesyncd

If you carefully observe kthread created many subprocess each with own motive. Through this you can carefully map the relation and treat processes accordingly during the time of production troubleshooting. Lets undersatnd and learns some trips and triks with this forest command in our next part of the blog. .