Back to projects

Transition File Processor

When a major vendor platform shift threatened to disrupt analytics processing, I built a custom file transformation system that bypassed 30+ hours of dev work, eliminated manual ops effort, and prevented long-term tech debt.

TypeScriptNode.jsXLSXPKGOps-first CLI Design

Context

Our shipping vendor, Pitney Bowes, announced a migration to a new analytics system that would change the data structure of their monthly reports. These reports fed into our enterprise systems but were manually massaged by an ops lead — ~4 hours of work each month.

The new format was incompatible with our legacy systems. Adapting internally would have required backend development, regression testing, and deployment coordination — eating up an estimated 30+ hours of senior developer time we couldn't afford to spend.

Problem

  • Two incompatible analytics formats — legacy system required both to be consumable, but couldn't handle them natively.
  • High fragility in manual process — each monthly file required ~4 hours of manual transformation by ops, including copy-paste reformatting and error-prone Excel operations.
  • Risk of downstream system breakage — incorrect formatting could cause ingestion failures, delayed transactions, or incorrect billing calculations.
  • Transition period overlap — both old and new formats had to be supported during vendor cutover, doubling surface area and increasing odds of human error.
  • No engineering time available — internal devs were fully allocated to revenue-generating systems. Building formal ingestion support would've cost an estimated 30–40 hours and introduced long-tail maintenance costs.
  • No error traceability — ops had no way to detect formatting issues until ingestion failed, making debugging reactive and brittle.

Intervention

I built a Node.js-based CLI tool that acts as a conversion proxy between the vendor's FTP drop and our system.

  • Parses new format (Pitney 360Ship)
  • Merges legacy-format exports if needed (--old flag)
  • Automatically creates per-meter and grand total rows
  • Applies pricing via hardcoded config
  • Generates error logs and exit codes for ops verification
  • Compiled to a standalone binary using pkg for zero-dependency deployment

The app validates structure, filters unnecessary records, applies surcharge logic, and produces a clean file consumable by our legacy system — no dev involvement, no manual editing.

Architecture Flow

Architecture flow diagram showing data transformation process

A unified pipeline transforming dual-format reports into a single, ingestible legacy output without modifying core systems.

Results

  • Conserved $4.5K+ in senior developer resources (30+ hours at $150/hr)
  • Automated $2K/year in manual operations labor (4 hrs/month at $40/hr)
  • Future-proofed ingestion for two report formats with zero additional dev cycles
  • Reduced error rate to zero (no user intervention needed)
  • Established self-reporting error logs for immediate troubleshooting

Leadership Feedback

This is an incredible solution — innovative and not a path most teams would take. It bridges the gap between legacy and the future.

— Internal feedback from leadership

Reflection

This project was my inflection point. Where scripting met systems design. I stopped thinking like a developer and started thinking like an architect.

I didn't just automate a workflow; I protected team velocity and operational continuity. That's what separates tools from infrastructure.