Energy data has a particular hazard: it always looks fine. A meter export opens in a spreadsheet, the numbers are plausible, the chart has the right daily shape. Nothing announces that the timestamps shifted by an hour in March, that one channel is in kW while its neighbour is in kWh, or that a sub-meter stopped reporting in week 14 and has been returning its last value ever since.
These faults do not stop a model from training. They produce a model that trains happily and is quietly wrong. This is the sequence of checks we run before treating a dataset as usable.
Why this comes first
The economics are lopsided. Running these checks on a year of half-hourly data is a day's work for someone competent with a spreadsheet or a short script. Discovering the same faults after a forecasting engagement has concluded costs the engagement, and usually the credibility of the next one.
There is a second reason. Several of these faults are worth fixing on their own merits, whether or not a model is ever built. A sub-meter that has been dead for eight months is a problem regardless.
Assumptions behind this method
- Interval metering data — typically 15-minute or half-hourly — from a building, site or process, exported as a time series.
- The intended use is forecasting, baselining, anomaly detection or optimisation. Billing validation has stricter requirements not covered here.
- At least twelve months of history, so that seasonal behaviour is observable.
- Thresholds below are working defaults from our own practice, not standards. They are starting points to argue with, and each states the reasoning so you can set your own.
The seven checks
1. Coverage
Sum your sub-meters and compare against the incoming supply. The unexplained remainder is the part of your consumption you cannot reason about.
If the main meter reads 100 and your sub-meters account for 55, no model will tell you much about the missing 45. Below roughly 70% coverage, we would treat sub-meter analysis as indicative only. The fix is metering, not modelling.
2. Resolution
| Question | Minimum useful interval |
|---|---|
| Monthly cost allocation | Daily |
| Load profile and baseline | Hourly or half-hourly |
| Peak demand management | Matching your tariff’s measurement window |
| Process-level anomaly detection | 1–5 minute, often faster |
The peak demand row is the one that catches people. If your charge is based on a 15-minute maximum, hourly data cannot see the quantity you are being billed for. Averaging has already removed it.
3. Timestamps
Four questions, in order of how often they find something:
- Timezone and daylight saving. Does the series handle the March and October transitions? A duplicated or missing hour twice a year is the classic signature. If the export is in local time with no offset recorded, you cannot reconstruct it.
- Interval convention. Does a reading stamped 14:00 describe the period ending or beginning at 14:00? Both conventions are common. Getting it wrong shifts everything by one interval, which is invisible in a chart and fatal in an event correlation.
- Monotonicity. Do timestamps ever go backwards or repeat outside the DST hour?
- Clock drift. Do devices agree with each other? Independent loggers with unsynchronised clocks drift by minutes over months.
4. Units and sign
Confirm for every channel: kW or kWh, and whether an interval value is an average power or an accumulated energy. Confirm the sign convention where export is possible — some meters report export as negative consumption, others on a separate register.
A quick sanity test: multiply average power by hours and compare to the billed total for a month. Agreement within a few percent means your units are probably right. A factor of two, or of four on 15-minute data, means they are not.
5. Gaps and frozen values
Missing intervals are easy: you can see them. The dangerous fault is the frozen channel, where a meter stops updating and repeats its last value. That looks like data rather than absence.
Test for runs of identical non-zero values longer than plausible. On half-hourly data, more than about six consecutive identical readings on a varying load is suspicious; on a genuinely constant baseload it may be normal, which is why the test needs a human verdict rather than a threshold alone.
Then decide the imputation rule before modelling and record it. Interpolating across a gap is defensible; interpolating across a shutdown invents consumption that never happened.
6. Drift and calibration
Compare each sub-meter's total against the main meter month by month. A ratio that is stable then trends is a calibration problem, not a behaviour change. Current transformers installed on the wrong conductor or with the wrong ratio produce constant proportional errors that survive every plausibility check.
7. Alignment with real events
Overlay production schedules, shift patterns, holidays and known shutdowns. Two failure modes show up here, and only here.
- Consumption that does not fall during a known shutdown — pointing at a frozen channel, a misattributed meter, or genuinely uncontrolled baseload.
- Consumption patterns that do not match the shift calendar — usually a timezone error from check 3, now visible because you have something external to compare against.
This check is the most laborious and finds faults the others cannot, because it is the only one that tests the data against the physical world rather than against itself.
A worked example
An illustrative case, with figures chosen to show the arithmetic rather than to describe a real site.
| Check | Finding | Consequence if missed |
|---|---|---|
| 1 Coverage | Sub-meters account for 68% of supply | A third of consumption invisible; savings claims unverifiable |
| 2 Resolution | Hourly data; demand charge on a 15-min peak | The billed quantity cannot be modelled at all |
| 3 Timestamps | Duplicate hour each October; convention undocumented | Two corrupted days a year, plus a possible one-interval shift throughout |
| 4 Units | One channel in kWh, others in kW | That channel over-weighted fourfold on 15-min data |
| 5 Gaps | Channel 7 frozen since week 14 | Model learns a constant; anomaly detection blind on that circuit |
| 6 Drift | Sub-meter 3 drifts from 0.31 to 0.24 of total | Apparent 20% “saving” that is a calibration fault |
| 7 Alignment | No drop during August shutdown on two circuits | Baseload misattributed; efficiency measures aimed at the wrong place |
Every one of these is individually mundane. Together they are the difference between an analysis that holds up and one that produces a confident number nobody can defend.
Reaching a verdict
Three outcomes, and all three are legitimate.
- Usable. Coverage adequate, timestamps understood, units confirmed, gaps documented with a stated imputation rule. Proceed, and record the rule alongside the results.
- Usable after remediation. Faults identified and correctable in the data — fix, document what you changed, then proceed.
- Not usable yet. Coverage too low or labelling too uncertain. The next investment is metering or recording practice, not analytics. This is a finding, not a failure.
Limits of this method
Billing disputes and formal measurement-and-verification claims impose stricter requirements than these checks. Treat what follows as a readiness screen rather than a metering audit, and do not rely on it where a standard applies.
It will not tell you whether your data supports a particular model either. It tells you whether the dataset describes reality closely enough for that question to be worth asking.
Found an error? Tell us and show your working — we correct in place and record what changed. Contact us.