Integrating With Customer Systems and Data
The pilot's data is spread across Harbourline. Carrier invoices arrive as PDFs in a shared AP mailbox, shipment records sit in the TMS, each carrier's agreed rates are in its own spreadsheet, and the depot gate log lands as a nightly file on an SFTP server. Every source has its own format and its own quirks. How you connect to them decides whether the pilot runs quietly for months or breaks every other morning.
The rule
Use the most stable door, keep quirks in one adapter, check everything entering.
- Use the most stable door. Prefer interfaces the owner supports and will keep: a documented API first, then a read-only database view agreed with the owner, then a scheduled file export. Scraping screens is the last resort, because it breaks whenever someone moves a button. Harbourline's TMS offers a read-only reporting copy of its database, so that is the door for shipments. The gate system offers only its nightly file, so for the gate log that is the door.
- Read from the source of truth. Shipment weights come from the TMS record, not from a dispatcher's spreadsheet copy that may be out of date.
- Keep quirks in one adapter. An adapter is the code that converts their format into your model. The rest of your system should never need to know that the TMS calls the lorry receipt number
LR_NO, pads it with spaces, or stores weight in tonnes. - Check everything entering. Validate required fields, types, units and dates at the boundary. A bad record goes to a quarantine list with a reason. It must not crash the batch or slip through.
def to_shipment(row: dict) -> Shipment:
"""One Harbourline TMS row -> one Loomwork Shipment."""
if not row.get("DISPATCH_DT"):
raise BadRecord("missing DISPATCH_DT")
return Shipment(
lr_number=row["LR_NO"].strip().upper(),
carrier=row["CARRIER_CD"],
weight_kg=float(row["WT_TONNES"]) * 1000,
dispatched=datetime.strptime(row["DISPATCH_DT"], "%d-%m-%Y").date(),
)Finally, make loads safe to repeat. Key each record on something unique, such as carrier plus LR number for a shipment, so that re-running yesterday's load updates records instead of duplicating them.
The shortcut
Trace one real record through every hop. Pick an actual invoice and follow it from where it enters to where it is checked. Each hop reveals a quirk the adapter must handle.
Where it enters
A PDF in the AP mailbox. The carrier printed the lorry receipt number as
lr7781, in lower case with a trailing space.
The trap is scattering customer fixes around the codebase: a trim here, a tonnes conversion there. Six months later nobody can tell which code exists only for Harbourline.
Drills
Answer fast — trust the rule.
Several ways into the TMS are available. Which should you prefer?
One shipment record arrives without a dispatch date. What is the best handling?
Why convert tonnes to kilograms inside the adapter rather than in the matching code?
Re-running yesterday's load doubled every shipment in Loomwork. What property was missing?
Which is the source of truth for a shipment's weight?
Remember
- Use the most stable door, keep quirks in one adapter, check everything entering.
- Shortcut: trace one real record through every hop.
- Trap: customer-specific fixes scattered across the codebase, and loads that duplicate data when re-run.