Why we chose Python for our automation scripts

Automation rarely begins with an ambitious platform. It starts with a repetitive task that has become too familiar: renaming a directory of files, checking a server log, preparing a release archive, or copying information between systems. In 2005 and 2006, many of our scripts grew from precisely those small irritations. We needed tools that could work across Windows and Unix machines, remain understandable after several months, and avoid turning a five-minute job into a maintenance project.

Python became our preferred answer because it occupied a useful middle ground. It was more structured than a collection of shell commands, less cumbersome than a general-purpose application framework, and more approachable than languages that demanded extensive boilerplate for modest jobs. The choice was practical rather than fashionable: we wanted dependable automation that another developer could inspect, alter, and run without a lengthy briefing.

Tool	Where it worked well	Where it became awkward	Our view
Shell scripts	Small Unix tasks and command pipelines	Portability, quoting, error handling	Excellent for narrow jobs
Perl	Text processing and established system utilities	Readability across a mixed team	Powerful, but uneven for shared scripts
VBA	Office documents and local spreadsheets	Server work and broader integration	Useful inside Microsoft Office
Python	File handling, APIs, testing, and cross-platform jobs	Deployment on machines without Python	The best general-purpose compromise

The work we needed to automate

Our scripts dealt with ordinary business machinery rather than laboratory problems. They gathered files from shared folders, checked naming conventions, generated reports, converted data between formats, and prepared packages for deployment. Some jobs ran on a Windows desktop; others ran overnight on a Linux server. A solution tied too closely to one operating system would have created another problem instead of removing one.

The Australian setting made this variety familiar. A team in Melbourne might be supporting a customer in Perth, with a three-hour time difference and an overnight process scheduled around the working day in both locations. A small office in Brisbane could have a Windows file server, a Linux development box, and a collection of old utilities that no one wanted to disturb. The script had to cope with real environments, not an idealised workstation.

This also influenced our attitude to complexity. Australian software shops often had compact teams where one person handled development, deployment, and support in the same week. There was little value in an automation language that required a specialist caretaker. A script needed to be readable by the person who inherited it during a busy arvo, with enough structure to reveal what it was doing without requiring archaeology.

Why Python fit the job

Python’s syntax made small programs look like the work they performed. A loop over files, a dictionary of configuration values, or a function that validated a record could be expressed without a wall of punctuation. Indentation was significant, which initially required discipline, yet that discipline gave the code a consistent visual shape. We could open an unfamiliar script and identify its main stages quickly.

The standard library was an important part of the decision. File paths, dates, regular expressions, compressed archives, email, subprocesses, and basic networking were available without assembling a large collection of third-party components. This reduced the number of moving parts in a script. For an internal utility, fewer dependencies meant fewer surprises when a machine was rebuilt or a package mirror was unavailable.

Python also allowed a script to grow without forcing an immediate rewrite. A ten-line file operation could become a command-line tool with configuration, logging, and tests while retaining much of its original structure. That progression mattered. Automation projects often begin as a quick fix, then acquire responsibilities gradually. Python gave us room to improve the fix rather than discard it.

The language was not perfect. Installing the right interpreter and libraries on Windows could be less straightforward than copying a batch file. Version differences mattered, and a script that ran beautifully on a developer’s machine still needed a sensible deployment arrangement. Even so, those costs were visible and manageable, while the costs of unreadable or fragile scripts tended to appear much later.

Readability was an operational feature

We treated maintainability as part of automation’s value, rather than as a separate concern reserved for large applications. A script that completes a task quickly but cannot explain a failure is only partially useful. Python encouraged us to separate input, processing, and output, to give operations meaningful names, and to put policy in configuration rather than burying it in a long command line.

That distinction became important when a script handled customer data or production files. A failed copy should identify the source, destination, and reason. A malformed record should be reported with enough context to locate it. A missing directory should produce a deliberate error rather than a cascade of confusing messages. Python’s exceptions and straightforward logging patterns helped us make those behaviours explicit.

Pair programming reinforced this benefit. Two developers could read a Python script together without spending the session translating syntax into intent. One person might notice an edge case in date parsing while the other questioned a file-deletion step. The language did not remove the need for review, but it lowered the cost of review. That was valuable in teams where developers moved between projects and could not remain experts in every utility.

The same quality helped with documentation. Comments were still necessary, but they did not have to explain every punctuation mark or shell expansion rule. A well-named function could carry much of its own description. In an archive of scripts, that small advantage accumulated: fewer handover notes, fewer mysterious workarounds, and less dependence on the original author.

Working across desktops and servers

A major attraction was the ability to use one language across different parts of the workplace. On Windows, Python could inspect directories, process Office-exported data, and invoke existing utilities. On Unix-like systems, it could read logs, manage scheduled jobs, and connect with the command-line tools already in use. The script did not erase platform differences, but it gave them a common layer.

That common layer was especially useful for release and deployment work. A script could assemble an application bundle, calculate checksums, copy selected files, and create a dated archive. It could verify each step rather than assuming that the operating system had done the right thing. When infrastructure later extended beyond local servers, the same preference for explicit, repeatable operations applied to hybrid cloud operations, where consistency matters across locations.

Portability still required care. File paths, line endings, permissions, character encodings, and available executables differed between systems. We learned not to claim that “cross-platform” meant “write once and forget”. It meant that the core logic could travel, while a small amount of platform-specific handling remained visible and testable.

Australia’s geography made this practical rather than theoretical. A support arrangement could span Sydney, Adelaide, and a remote site with a slower link or limited local expertise. A script that performed its work locally, recorded useful results, and returned a compact status report was preferable to a process dependent on constant interactive access. Reliable automation helped compensate for distance without pretending that distance did not exist.

Testing small tools seriously

It is tempting to think that a short script does not need tests. We found the opposite. A small utility often handles files that cannot easily be recreated, runs at inconvenient hours, or makes changes that are difficult to reverse. A modest set of tests can protect the assumptions that matter most: date formats, empty input, duplicate records, missing directories, and unexpected characters.

Python made unit testing accessible without imposing a large framework. We could test a parsing function separately from the file system, pass sample data to a transformation routine, and verify that an error was raised for invalid input. This separation encouraged better design. Code written to be tested was usually code with clearer boundaries and fewer hidden dependencies.

For scripts that changed files, we used temporary directories and dry-run modes where possible. A dry run showed which files would be copied, renamed, or removed without performing the final action. Logs recorded counts and destinations, while exit codes allowed scheduled jobs to signal failure to another system. These details turned a script from a personal convenience into a dependable part of operations.

The local commercial context mattered here as well. Australian organisations frequently had to balance useful controls with modest budgets, particularly outside the largest financial and government environments. A well-tested Python utility could provide a measure of operational discipline without requiring a large automation suite. It was a practical fit for teams that needed reliability but could not justify a separate platform for every workflow.

Choosing a tool without making it a religion

Python did not replace every other scripting tool. Shell remained the right choice for a short Unix pipeline. Batch files were sometimes the simplest way to launch a Windows command. VBA remained useful when the data lived inside a spreadsheet or an Access database. The decision was to choose the smallest tool that kept the job clear, not to force every task into Python.

Our preference emerged where several concerns met: cross-platform behaviour, structured data, repeatable file operations, error handling, and a likely need for future maintenance. If a script was expected to grow, be shared, or run unattended, Python usually offered a safer starting point than a dense command sequence. If it was disposable and local, a shell command could be entirely appropriate.

There was also a human reason for the choice. Developers are more likely to improve automation they can understand. A Python script invited refactoring, logging, tests, and configuration because those changes felt proportionate to the task. A tangled collection of commands often encouraged the opposite response: leave it alone, because touching it might reveal another dependency.

Over time, this shaped our wider development practice. Automation became something we could review, version, test, and hand over rather than a private trick stored on one workstation. Python gave us a stable centre for that work, while still allowing other tools to remain where they made sense. The lasting decision was less about loyalty to a language than about making routine computing work visible, repeatable, and safe to change.