Writing Command-Line Tools With LLMs in November 2026: Write the Help Text First, Keep Output Pipe-Friendly, and Test Every Exit Code

Small command-line tools are one of the best jobs to hand a language model. A script that renames files, pulls a report, or checks a config is short, self-contained, and easy to describe. That is also why so many of them turn out badly: the model produces something that works once on your laptop, prints a cheerful message on failure, exits with status 0 anyway, and breaks the moment someone pipes it into another command. This guide shows how to get a tool you can actually rely on, using Python as the running example.

The short version: design the interface before any code exists, keep data on standard output and everything else on standard error, and do not ship until you have tested what the tool does when things go wrong.

The terminal has changed a lot since the VT100, but the contract has not: data on standard output, messages on standard error, and an exit code that tells the truth. Image: DEC VT100 Terminal by Wolfgang Stief, CC0, via Wikimedia Commons.

Start with the help text, not the code

The interface of a CLI is its contract. Flag names, argument order, defaults, and exit codes are what other people and other scripts depend on, and they are painful to change later. So write the --help output yourself, or have the model draft it, and agree on it before asking for an implementation.

A useful first prompt looks like this:

I want a Python 3.12 command-line tool called csvdedupe.
Draft ONLY the --help output, no code yet.

What it does: reads a CSV file (or stdin), removes duplicate rows
based on one or more key columns, writes the result to stdout.
Who runs it: me in a terminal, and a nightly shell script.
Must support: choosing key columns, keeping first or last duplicate,
a --dry-run that only reports how many rows would be removed.
Must not: modify the input file in place.

Read the draft as if you were a new user. Are the flag names obvious? Does every option have a sensible default? Is there anything you would have to guess? Push back on the things models like to add for free, such as a --verbose level system with five settings, a config file, or color output you never asked for. Every extra flag is something to test and maintain.

Once the help text is right, paste it back and say: "Implement exactly this interface using argparse from the standard library. Do not add flags that are not in the help text." Naming the library matters. If you do not, you may get a third-party framework you then have to install everywhere the script runs.

Keep data and messages apart

The most common flaw in generated CLI code is printing everything with print(). That sends status lines like "Processing 4,000 rows..." to standard output, mixed in with the actual data. It looks fine in a terminal and corrupts the output the first time you write csvdedupe users.csv > clean.csv.

The rule to give the model is simple:

• Standard output carries the result, and only the result.
• Progress, warnings, and errors go to standard error, for example with print(..., file=sys.stderr) or the logging module, which writes to stderr by default.
• Do not print a success message to stdout. Silence plus exit code 0 is the success message.

If the tool produces something another program might read, ask for a machine-readable option such as --format json, and make sure the JSON output is a single valid document, not JSON lines mixed with human text.

Handle a closed pipe

Run your tool as csvdedupe big.csv | head -n 5. A naive Python script will often print a BrokenPipeError traceback when head stops reading. That is noise, not a real failure. Ask the model to handle it, and check that it does. A common pattern is to catch BrokenPipeError around the main output loop, redirect stdout to os.devnull so the interpreter does not complain again while flushing at exit, and exit quietly. The Python documentation for the signal module describes this case and this approach, so you can compare the model's version against it.

Make exit codes mean something

Shell scripts, cron jobs, and CI pipelines decide what to do next based on the exit code. A tool that always exits 0 makes every failure invisible. Write the exit codes down before implementation, in the help text or a short table:

• 0: success.
• 1: the tool ran but hit an error, such as an unreadable file or a missing key column.
• 2: bad usage, such as an unknown flag. argparse already exits with 2 when parsing fails, so keep that convention.

If you need a code for "it worked, but found something", the way grep exits 1 when nothing matches, decide that explicitly and document it. Then check the generated code for two things models do often: a bare except Exception: print(e) that swallows the error and falls through to a normal exit, and a main() function whose return value is never passed to sys.exit().

Validate input early and say exactly what is wrong

When a key column is missing, a good tool says so in one line that names the column and the file, then exits 1. A poor one gets halfway through, raises a KeyError traceback, and leaves a partial output file behind.

Ask the model to:

1. Check that input files exist and are readable before doing any work.
2. Check that requested columns are present in the header, and list the columns that are available when one is missing.
3. Print error messages without a traceback in normal use, but keep tracebacks available behind a --debug flag or an environment variable, because you will want them when something truly unexpected happens.

Every extra flag is another blade. The ones that can cut, like writes and deletes, deserve the closest look. Image: Swiss army knife open by Jonas Bergsten, Public domain, via Wikimedia Commons.

Be careful with anything that writes or deletes

If the tool changes files, it needs extra care, and this is where you should read every line the model wrote.

• Dry run first. A --dry-run flag that prints what would happen, to stderr, without touching anything. Test it by running it and then confirming with git status or a file listing that nothing changed.
• No silent overwrites. Refuse to overwrite an existing output file unless the user passes something like --force.
• Write atomically. Write to a temporary file in the same directory, then rename it over the target with os.replace(). That way a crash halfway through never leaves a half-written file where the good one used to be.
• Watch paths. If the tool builds paths from user input or from file contents, check for .. components and absolute paths that escape the intended directory.

Also check how the code runs other programs. If it uses subprocess, it should pass a list of arguments, not a string with shell=True. A filename with a space or a semicolon in it should be treated as a filename, never as part of a command.

Test the interface, not just the functions

Models will happily write unit tests for the inner deduplication function. Those are useful, but they do not tell you whether the tool behaves correctly as a command. Ask for tests that run the CLI the way a user would, either by calling main(argv) directly with a list of arguments or by running it with subprocess.run and checking the result.

A small but solid test list for a tool like this:

1. Happy path: known input file, known expected output, exit code 0, nothing on stdout except the data.
2. Reading from stdin instead of a file gives the same result.
3. A missing key column exits 1, prints one clear line to stderr, and writes nothing to stdout.
4. An unknown flag exits 2.
5. A nonexistent input file exits 1 with a message naming the file.
6. --dry-run leaves the filesystem unchanged.
7. Piping into head -n 1 produces no traceback.
8. An empty input file and a header-only input file both behave sensibly.

Run the tests yourself. Then do the step people skip: break the code on purpose, for instance by changing an exit code from 1 to 0, and confirm a test fails. If nothing fails, the test is not checking what you think it is.

Check the help text one last time

After a few rounds of changes, the help text and the behavior drift apart. A flag gets renamed in the code but not in the description, or a default changes. Finish by running --help, reading it next to your original draft, and trying each documented example by hand. If an example in the help text does not work when pasted into a terminal, fix the tool or fix the text before anyone else sees it.

A workflow you can reuse

1. Describe the job, the users, and what the tool must not do. Ask for help text only.
2. Edit the help text until a stranger could use the tool from it alone. Write down exit codes.
3. Ask for an implementation of exactly that interface, using the standard library unless you have a reason not to.
4. Review for stdout and stderr separation, real exit codes, early validation, and safe file writes.
5. Ask for command-level tests, run them, and break the code once to prove they can fail.
6. Try it in a pipe, with bad input, and with --dry-run before you point it at real data.

None of this is unique to working with a model. These are the habits that separate a throwaway script from a tool other people can trust. The model just makes the first draft fast, which gives you the time to do the checking properly.