3 Billion Records and a 52-Hour Job: What a Sub-Penny Transaction Taught Me About D365 at Scale

I want to tell you about the most interesting problem I've ever had to solve in 13 years of working with Microsoft Dynamics.

Not the most glamorous. Not the most high-profile. But the one that taught me the most about what happens when enterprise software meets enterprise-scale data and what you must be willing to do to fix it.

The Scene

I was brought in as part of a specialist task force to support a company, one of Europe's largest online travel platforms, supporting over 8,000 users across 30 countries. The engagement came through Microsoft Partner Engage Group, and from the moment I arrived, it was clear this wasn't a routine support job.

The finance team had a problem. Two of their most critical month-end processes, settlement reconciliation and monthly revaluation, had effectively stopped working.

Not crashed. Not errored out. Just... stopped being viable.

Reconciliation jobs were running for over 52 hours. Revaluation jobs for over 54 hours. In a business operating across 30 countries with constant transactional volume, that meant month-end close was becoming operationally impossible. Finance teams were exhausted. Deadlines were being missed. Pressure was building from the top.

The system wasn't broken. But it was drowning.

The Investigation

The first thing I did was resist the instinct to look for a quick fix.

When a system is performing this badly, there's always a temptation to patch to tweak a batch job setting, increase a timeout, throw more infrastructure at it. These fixes feel productive. They rarely solve anything.

Instead, I went looking for the root cause.

The D365 database at thecompany was carrying over 3 billion records. That's not unusual for a business of this scale, years of transactional data across dozens of entities, currencies, and markets accumulate fast. But 3 billion records is also a lot of places for a problem to hide.

I started pulling data. Analysing transaction volumes. Looking at what the reconciliation engine was trying to process and where it was getting stuck.

And then I found it.

The Insight

Buried within the database were approximately 800,000 unsettled transaction balances.

Each one was under £0.10.

Sub-penny amounts. Rounding differences. The kind of tiny variances that accumulate silently over years in high-volume transactional environments, currency conversions that don't quite round cleanly, payment processing fees that leave microscopic remainders, settlement differences so small that no human would ever notice them on a report.

Individually, they were meaningless. Collectively, they were catastrophic.

Every time the reconciliation engine ran, it was attempting to process and settle each of those 800,000 items alongside millions of legitimate transactions. The sheer volume of low-value items was creating a bottleneck that the system simply couldn't clear in any reasonable timeframe.

The database wasn't too big. It was too cluttered.

The Decision

Here's where it got interesting because the technical diagnosis was actually the easy part.

What I had to do next wasn't a system configuration change. It wasn't a code fix or a module update. It was a business decision.

I had to make the case to write off 800,000 transactions.

Now, before anyone panics the total value of those 800,000 balances was financially immaterial. We're talking about amounts so small they wouldn't move the needle on any meaningful financial report. But in an enterprise finance environment, writing off transactions even tiny ones requires sign-off, justification, and confidence.

I put together the analysis. I walked the stakeholders through the data. I explained what these balances were, why they existed, what they were costing the business in system performance, and what the financial impact of writing them off would be which was, essentially, nothing. The decision was made. The write-off was executed. And then we waited.

The Result

The next reconciliation job ran in 4 hours.

Revaluation dropped from 54 hours to 5 hours.

Unreconciled transactions fell from over 3 million to 1.2 million.

Month-end close became viable again. The finance team got their evenings back. The pressure from the top evaporated. It sounds almost anticlimactic when I put it like that 800,000 tiny numbers causing 50 hours of pain. But that's often how it works at scale. The biggest performance problems aren't caused by big things. They're caused by small things that nobody thought to look at, multiplied by millions.

What This Taught Me

Thirteen years into working with Dynamics, this engagement reminded me of a few things I think every D365 consultant needs to keep front of mind:

1. Scale changes everything. A process that works perfectly at 10 million records can fall apart at 3 billion. Never assume that because something works in a mid-size environment it will hold up at enterprise scale. Always ask about data volumes before you start diagnosing.

2. The technical fix is rarely just technical. The write-off decision required stakeholder buy-in, financial justification, and clear communication. If I'd gone in with just the technical analysis and no business case, nothing would have moved. The best consultants are translators they speak both languages.

3. Root cause over remedy. Every hour I spent investigating before touching anything saved days of remediation later. The temptation to act fast is always strong when pressure is high. Resist it. Understand the problem first.

4. Sub-penny transactions are not a company problem. They accumulate in every high-volume D365 environment over time. If you're running a large finance operation on D365 and you've never audited your unsettled low-value balances it might be worth a look.

A Final Thought

The engagement is one I think about often not because it was the most complex thing I've ever done technically, but because it's a clean example of what good consulting looks like.

You show up. You resist the easy answer. You dig until you find the real problem. You build the case. You make the call. And then you watch 52 hours become 4.

That's the job.

Seth Campbell is a D365 F&O Functional Consultant with 13 years of Dynamics experience across finance, supply chain, and systems administration. He is also the founder of LaunchPad D365, a coaching platform helping aspiring consultants break into the D365 market.

If you're working through a complex D365 challenge or want to connect, feel free to reach out.