Why thinking about data from the product design stage can determine what an organization will be able to learn in the future.

At a company where I worked, we had an application process that moved through different states.

Every time an application progressed, the system updated its state. If it was started and moved to validation, we simply replaced one value with another.

Operationally, it worked. The system always knew the current state of each application, and that was enough to keep the process moving.

Until a fairly simple business question came up.

We wanted to understand how long customers took to move from one state to the next. We wanted to identify where they were getting stuck so we could evaluate whether an intervention, a reminder, or some other kind of nudge could help them complete the flow.

And we couldn't answer it.

We knew where each application was, but not how it had gotten there. The previous state had been overwritten. We didn't have timestamps for each transition or the history needed to reconstruct the journey.

At that point, it was no longer an analytics problem.

The data simply didn't exist.

Saving the current state allows us to operate. Saving the history allows us to learn.

A system can work perfectly and still lose valuable information

When we design software, we naturally think first about what the system needs to do.

What information do we need to execute this process? What state is an application in? What is the balance of an account? Which product did a customer choose? What was the outcome of a transaction?

These are necessary questions. The problem appears when we design exclusively to answer them.

This tension between operating in the present and preserving history is well documented in architectural patterns such as Event Sourcing. In a traditional model, we usually update the current state of an entity and lose its history unless we explicitly add auditing mechanisms. Event Sourcing, on the other hand, preserves the sequence of events that produced that state.

I don't believe every system should implement Event Sourcing. It introduces complexity and trade-offs that aren't always justified.

But I do believe every system should ask one question before deciding what information to preserve:

What will we want to understand tomorrow about what is happening today?

Because these are two different problems.

A system can be perfectly designed to execute an operation and poorly designed to learn from that operation.

Time is part of the data too

There is one thing I try to pay particular attention to when thinking about data: time.

Suppose we have this information:

status = approved

That may be enough to execute the process.

But to analyze it, we may need to know much more: when it was approved, how long it remained in the previous state, what happened before approval, how many times its state changed, which rule was active when the decision was made, or whether there was a manual intervention.

Many of the interesting questions we ask about a business aren't just questions about what happened.

They're questions about when it happened, in what order, and under what context.

In data management, there is a particularly useful concept: data provenance. It refers to preserving information about the origin of data and the transformations it went through during its lifecycle.

In a digital product, we don't necessarily need to apply this concept with the same level of rigor as in a scientific environment, but the underlying idea is powerful:

Your system shouldn't just record what happened. It should allow you to understand how it happened.

Because once we lose the sequence, we may still have the outcome while having lost a fundamental part of the information.

The most expensive data may be the data you decided not to save

Storing data today is relatively inexpensive.

That doesn't mean data is free.

There are costs associated with processing, pipelines, quality, observability, governance, security, privacy, and querying. The more data we preserve, the more we have to manage.

That's why I don't believe in saving absolutely everything "just in case." We have to make decisions.

What's interesting is that those decisions are asymmetric.

If we save a piece of data and discover three years later that we never used it, we probably incurred an unnecessary cost. But if we decide not to save it and discover three years later that it was essential to answering a question, there may be no way to recover it.

We can decide to delete information later. What we can't always do is go back and capture something we never recorded.

The most expensive data may be the data you decided not to save.

Not because every piece of data will eventually be valuable.

But because some opportunities to capture it happen only once.

The problem is that we don't yet know what we'll need

This comes up all the time in Data Science.

Someone might ask:

"Will this variable actually be useful for the model?"

And very often, the correct answer is:

We don't know.

We may have an intuition. Based on experience, we may believe that a particular variable should be related to the behavior we're trying to predict, but we need to test it.

If the data already exists, we can analyze it, build the variable, evaluate its predictive power, and decide whether it's worth incorporating. If we can reconstruct it retrospectively from another source, that's another option.

The problem appears when the data doesn't exist and can't be inferred.

Then we need to modify the system to start capturing it and wait until enough history has accumulated. That may take weeks, months, or even longer, depending on the phenomenon we're trying to study.

So missing data doesn't just create an analytical limitation.

It can create waiting time before we're able to make a decision.

And that cost can be far greater than preserving a variable or event that we ultimately never use.

Data should be there from the beginning of product development

That's why I try to get development teams to think in a data-driven way too.

Traditionally, we might imagine a flow like this:

Business defines → Development builds → The system generates data → Data analyzes

The problem is that when Data only appears at the end, many decisions have already been made: which events are recorded, which timestamps exist, what information gets overwritten, which identifiers allow systems to be connected, what context we preserve around a decision, and what history we can reconstruct.

Later, we can build an extraordinary pipeline, an impeccable warehouse, and highly sophisticated dashboards.

But none of those things can invent information that the product never generated.

That's why I prefer to think about the process differently:

Business + Development + Data design → The product operates → The product generates knowledge

This doesn't mean Data needs to approve every feature or that we should add bureaucracy to development.

It means introducing a few questions from the very beginning:

  • What do we want to measure?
  • What might we want to understand later?
  • What future decisions could this feature inform?
  • What history do we need to preserve to reconstruct what happened?

Data infrastructure begins long before the data warehouse.

It begins in product design.

Designing for learning doesn't mean saving everything

There is an important balance here.

We could take this argument to the extreme and conclude that we should save absolutely everything.

I don't think that's a good solution.

Beyond processing and governance costs, there is data we shouldn't retain indefinitely for privacy, security, or regulatory reasons. And the more data we accumulate, the greater our responsibility for it becomes.

The goal isn't to maximize the amount of data we collect.

It's to improve our ability to make good decisions about what is worth capturing, how long we should keep it, and at what level of detail.

At the beginning, there will probably always be some degree of intuition involved. We don't know every question the business will ask two years from now.

But we can pay particular attention to information that is difficult or impossible to reconstruct later: state changes, events, timestamps, decision context, versions of rules or models, and relationships between entities.

We can decide later what to preserve and for how long.

Capturing and retaining don't necessarily have to be the same decision.

AI makes this discussion even more important

We're also entering a particularly interesting moment.

We have increasingly powerful tools for processing large volumes of information, finding patterns, and working with less structured data. At the same time, coding agents and vibe coding are dramatically reducing the effort required to build certain features.

That can lead to an interesting consequence.

If we need to spend less energy mechanically writing every line of code, we can spend more energy thinking about what we're building, why we're building it, and what we want to get from it.

That's why I believe specs are going to become increasingly important.

A specification could simply say:

When X happens, change the state to Y.

But a good spec should take us a little further.

What event do we want to record? When did it happen? What triggered the transition? What information do we need to preserve? How will we measure whether the feature produced the expected outcome? What will we want to analyze about this process six months from now?

AI can help us build faster.

But building faster also makes it more important to decide what knowledge we want our systems to produce.

Designing software also means designing evidence

In the previous article, I argued that being data-driven doesn't mean having an answer for everything.

It means building a method that allows us to formulate hypotheses, test them, and quickly discover when we're wrong.

But there is a prerequisite for doing that:

we need evidence.

If we want to experiment with a process, we need to be able to measure it. If we want to understand behavior, we need to have captured its signals. If we want to compare one decision with another, we need to know what happened at each point in time. If we want to train a model, we need history.

That's why I increasingly think of data design as something that happens before, not after, product development.

Designing software also means designing the evidence we'll have tomorrow to understand whether that software actually worked.

And that changes where a data strategy should begin.

It doesn't begin when we connect the product to the warehouse.

It begins when we decide what the product will record.

The history we didn't save

Let's go back to the example from the beginning.

We had the applications. We had their current state. We had a working system.

What we didn't have was the history.

And without that history, we couldn't answer a question nobody had anticipated when the system was built.

We could modify it to start recording transitions from that point forward. What we couldn't do was perfectly reconstruct everything that had already happened.

That experience left me with a question I now try to ask much earlier in the development process:

What will we want to learn from this after it has been running for a year?

We won't always get it right.

We'll save things we never use, and we'll inevitably forget things we later wish we had captured.

The goal isn't to predict every future question.

It's to recognize that our ability to learn tomorrow is also designed today.

We can change an architecture. We can refactor code. We can replace a technology.

But there is something much harder to recover:

The history we decided not to save.

Sources and further reading

  • Microsoft Azure Architecture Center — Event Sourcing pattern. A reference on preserving change history through events and the trade-offs associated with this architectural pattern.
  • NIST Research Data Framework (RDaF). A framework covering the data lifecycle, metadata, provenance, preservation, use, and reuse.
  • NIST — Data Provenance. The concept of data provenance and preserving information about the origin and transformations of data.