os and bufio

The os package is Go's primary interface to the operating system. Files, directories, environment variables, process information, and command-line arguments all flow through it. The bufio package builds on top of os (and the broader io.Reader / io.Writer model) by adding buffering — turning many small reads and writes into fewer, more efficient system calls. Together, these two packages cover most of what programs need to interact with their environment.

Opening and reading files

The simplest way to read a file is os.ReadFile. It opens the file, reads the entire content into a byte slice, and closes the file — all in one call:

os.ReadFile is the right choice for small files where you need the content all at once. For large files or cases where you want to process content as it arrives, you need more control.

os.Open returns an *os.File that implements io.Reader. This lets you pass it directly to any function expecting a reader, wrap it in a buffered reader, or process it in chunks:

Always call defer f.Close() immediately after the error check. Open files consume operating system resources (file descriptors), and there is a per-process limit on how many can be open at once. Forgetting to close a file is a resource leak that will eventually cause the program to fail.

Writing files

os.WriteFile creates or truncates a file and writes a byte slice to it in one call:

The third argument is the file permission bits. 0644 means the owner can read and write; everyone else can only read. This is the conventional default for data files.

When you need more control — creating a file only if it does not already exist, or appending to one — use os.OpenFile with explicit flag combinations:

The flags compose with bitwise OR. The most common ones are:

Buffered I/O with bufio

Every call to Read or Write on an *os.File is a system call. System calls are expensive — they require the CPU to switch from user mode to kernel mode and back. When your code reads one byte at a time or writes many small strings, this overhead dominates.

The bufio package solves this by wrapping a reader or writer with an in-memory buffer. Reads are satisfied from the buffer until it is empty, then refilled in one large system call. Writes accumulate in the buffer until it is full or explicitly flushed.

Reading line by line with bufio.Scanner

bufio.Scanner is the idiomatic way to iterate over structured text. Its default split function breaks input at newlines:

scanner.Scan() advances to the next token. When there is nothing left to read, it returns false. scanner.Text() returns the current token as a string with the delimiter stripped. Always check scanner.Err() after the loop — if Scan returned false due to an error rather than EOF, Err() reports it.

You can change how the scanner splits input by calling scanner.Split before the loop. The bufio package provides four built-in split functions:

Fine-grained control with bufio.Reader

When you need more than line scanning — reading up to a delimiter, peeking ahead, or unreading a byte — bufio.NewReader gives you those capabilities:

ReadString returns everything up to and including the first occurrence of the delimiter. If the source ends before the delimiter is found, it returns whatever it has along with io.EOF.

Other useful methods on bufio.Reader:

Buffered writing with bufio.Writer

bufio.NewWriter wraps an io.Writer and accumulates writes in memory, reducing the number of system calls:

The critical step is Flush. A bufio.Writer holds data in its internal buffer until the buffer is full or you explicitly flush it. If you forget Flush, the last chunk of data may never reach the underlying writer, leaving your file incomplete.

Environment variables

Environment variables are a standard mechanism for configuring programs without hardcoding values into the binary. They are especially common in containerized and cloud environments. The os package provides three functions for working with them.

Reading with os.Getenv

os.Getenv returns the value of a named variable:

If the variable is not set, Getenv returns an empty string. This creates an ambiguity: an empty result might mean the variable is missing, or it might mean the variable was explicitly set to an empty string.

Distinguishing unset from empty with os.LookupEnv

os.LookupEnv resolves the ambiguity by returning a second boolean value:

ok is true only when the variable exists in the environment, regardless of its value. Use LookupEnv whenever you need to distinguish "not configured" from "explicitly empty."

Setting and removing variables

os.Setenv sets a variable for the current process:

The change is visible to subsequent Getenv calls within the same process and to any child processes spawned after the call. It does not affect the parent process or persist after the program exits.

os.Unsetenv removes a variable:

To read all variables at once, os.Environ returns a snapshot of the environment as a []string slice where each element has the form "KEY=VALUE".

Command-line arguments

Every Go program can access its raw command-line arguments through os.Args:

os.Args is a []string. os.Args[0] is always the path to the executable. The arguments that the user typed start at index 1. Running ./myapp --port 8080 --debug would produce:

Before indexing into os.Args, always check its length to avoid a panic:

os.Exit(1) terminates the process immediately with a non-zero exit code, signaling failure to the shell or calling process. Note that os.Exit does not run deferred functions, so it should only be used at the outermost level of main.