From Samples to Data Streams

Internal audit methodology was built for a world of periodic release cycles and manual evidence collection. A traditional sample of 30 to 50 items made sense when systems changed slowly. It does not hold up against a company doing 250,000 deployment events a year, where that same sample covers 0.02% of the population.

The talk works through four signals that a control environment has outgrown sampling: population scale, controls that change mid-period, teams implementing the same control objective differently, and evidence scattered across systems. For each, it shows what changes when audit moves from a point in time sample to testing defined conditions across the full population: outliers surface that aggregate results conceal, including a static analysis gate left in advisory mode on one pipeline, and an in-house approval workflow that let people approve their own changes.

The argument is not that sampling is obsolete. It is that in high frequency delivery environments, sampling needs to be complemented by analytics, continuous monitoring, and evidence collected at the point of change, so audit can answer “were controls effective throughout the period” instead of only “do they look effective today.”