The text log is the format that feels natural and serves nobody but the person who wrote it. The line reads like a sentence, timestamp, level, module, message, and the sentence answers the question its author imagined, which is the last question it will ever answer well. The grep that follows is the archaeology, the engineer typing the substring they remember, hoping the message's phrasing survives the deploy that rewrote it, joining the request's scattered lines by eye, and reconstructing the incident from prose. The structured log inverts the contract, the line is data, the fields are named, and the question is a query rather than a memory. This article builds the structured practice properly, the event model that decides what a log line is, the field discipline that keeps years of logs queryable, the collection pipeline that ships them, and the honest comparison with the grep era, including the cases where the text line still wins.
The event model, one line one fact and fields for what varies
The structured log's unit is the event, and the event discipline starts with the sentence a log line must finish, something happened at a time to a thing with an outcome. The line is a JSON object with the named fields, the timestamp and level, the service and instance, the event type and the outcome, and the identifiers that tie the event to its context:
{"ts": "2026-10-17T09:41:22.418Z", "level": "warn",
"service": "checkout", "event": "payment_retry",
"trace_id": "4bf92f3577b34da6", "order_id": "ORD-44172",
"attempt": 2, "gateway": "primary", "outcome": "timeout"}The field that separates the structured log from the decorated text log is the event type, the closed vocabulary that the team owns, payment_retry not the message string payment attempt timed out, and the vocabulary is the queryability's foundation, the query for event:payment_retry works across years of deploys because the event name is a contract rather than a phrase. The rule that keeps the model honest is one line one event, the multi fact line, payment failed and also retried and also queue is degraded, is three events that grep and query both handle badly, and the event that spans lines, the stack trace, arrives as a field of its event rather than as twenty pseudo events.
The field discipline and the cardinality ceiling
The fields fall into two classes with different rules. The dimension fields, the service, the environment, the event type, the outcome, are the closed vocabularies, documented, enumerated, and safe to aggregate on, and they are the query's indexing surface. The context fields, the identifiers, the timings, the quantities, are the open values, the trace identifier, the order number, the latency in milliseconds, and they answer the drill down but never the aggregate. The discipline's one law is the cardinality ceiling, the dimension fields stay bounded, the context fields are never indexed as dimensions, the order identifier is a filter value not a facet, because the log store that indexes a million order numbers is the store whose index grows with the business until the query and the invoice both fail.
# the field contract, versioned with the code
event: payment_retry
dims: [service, environment, gateway, outcome, attempt]
context: [trace_id, order_id, latency_ms]The correlation identifiers are the field discipline's highest value members, the trace identifier from the tracing work crossing into every log event on the request path, and the request's whole story becomes one filter, trace_id, the structured log's answer to the grep join the text era never finished. The personal data rule completes the discipline, the fields that would identify a person are not logged, the identifier is hashed or omitted, and the rule is a code review item rather than a compliance afterthought, because the log store is the widest audience the data will ever have.
The collection pipeline and the store that answers
The shipping path is the pipeline, the application writes to stdout by the container contract, and the collector, Filebeat, Vector, Fluent Bit, does the reading, the enrichment and the shipping. The enrichment is the collector's quiet value, the fields the application should not know, the infrastructure tags, the region, the cluster, the instance profile, added at the boundary where they are true, and the pipeline's shaping rules, the parse of the legacy service's text format into the structured contract, the field name normalization, all live in the collector config, versioned, reviewed, one place.
The store answers the queries, and the honest menu has three positions. Elasticsearch, the full text engine with the aggregation power, the store that the article's title names, and the trade is the operational surface, the cluster that the capacity planning must tend, the index lifecycle that the retention needs, the bill that the ingestion rate drives. Loki, the label indexed store, the cheaper shape that indexes the dimensions and compresses the content, the right store for the queries that filter and read rather than the queries that aggregate everything. The cloud log services, the managed stores with the least operational surface and the highest per gigabyte bill, and the choice among the three is the same volume conversation the pipeline articles keep having, the ingestion rate per day, the retention per compliance, the query surface the team actually runs, priced against the operational hours each option bills.
The query practice that replaces the archaeology
The structured practice's payoff is the query workflow during an incident, and the difference from the grep era is the difference between asking and remembering. The question that opens the investigation is the filter, the trace identifier from the user's report, and the timeline arrives complete, every event on the path, ordered, and the first answer is the whole story rather than the first guess. The second question is the aggregate, count by event the errors in the window, and the shape of the incident appears, the payment_retry count that spiked at 09:40 while the gateway timeouts spiked beside it, the correlation the text log would have produced from an engineer's intuition and an hour of joins:
event.dataset:checkout AND event:payment_retry
AND outcome:timeout AND @timestamp:[2026-10-17T09:00:00Z TO +1h]The third question is the longitudinal one, the query that no grep can run, the rate of payment_retry by day for the month, the trend that says whether Tuesday was an incident or a pattern, and the structured store's aggregation over the named events is the product feature the text log never had. The practice's alerting rides the same queries, the alert on the event rate's anomaly, the structured log's event vocabulary closing the loop that the metrics discipline opened.
One incident opened both ways, the quarter hour that proves the contract
The incident makes the comparison concrete. The report arrives at 09:52, checkout is failing for some customers, and the engineer sits down at 09:53. The structured session, the report contains the order number, the first query is the order's events, order_id:ORD-44172, and the timeline returns the payment_retry events, attempt two at 09:40, the outcome timeout, the gateway field primary, and the trace identifier, which is the second query, the full request path, and the 09:58 conclusion is the causal shape, the timeouts correlate with the primary gateway, the retry against the secondary succeeded. The third query at 10:00 is the aggregate, the payment_retry rate by gateway for the hour, the primary's spike against the secondary's flat line, and the 10:04 action is the routing decision, the status page update at 10:05. The structured session's total is one engineer, twelve minutes, three queries.
The grep session opens at the same minute with the same report, and the first command is the substring guess, grep "ORD-44172", across the rotated files of the three services, and the results need the eye, the lines the deploy two weeks ago rewrote, the message phrased differently in the gateway service, the join by hand, the timestamps that the services formatted differently. The pattern emerges around 10:10 from the engineer's experience rather than from the query, and the aggregate question, how widespread, is the command they type five ways before one of them returns something interpretable, and the honest grep session's total is one senior engineer, the system knowledge that just left the company's other half, and twenty five minutes of reconstructed prose. The same incident, the same platform, and the difference is the contract, the named fields that the query trusts against the remembered phrases that the grep hopes.
Levels, sampling and the retention economics
The level vocabulary is the discipline's coarse channel, and the working rules are short. Error pages, the event that means the user's request failed, warn is the degraded path, retry, fallback, the operation that succeeded uncomfortably, info is the business event, the order placed, the payment settled, and debug is the local tool that production never emits, and the level's contract is the alerting surface, error and warn feed the alerts, info feeds the metrics derivation and the audits, and the level inflation, the info logged as error, is the alert fatigue's root cause. The sampling question is the ingestion's economics, the debug that never ships, the info sampled at high volume, the error always shipped whole, and the sampling decision is per event type with the compliance and the debugging needs priced against the ingestion line.
The retention economics close the practice, the retention tiers that the stores support, the hot days for the incident queries, the warm months for the patterns, the cold archive for the compliance calendar, and the policy written per event class, the audit events outliving the debug forever. The invoice's honest line is the ingestion rate times the retention, and the discipline's fields are the invoice's levers, the level discipline that keeps the info volume bounded, the sampling that trims the high volume events, and the retention that the compliance calendar rather than the comfort zone sets.
Where the text log still wins and the honest boundary
The comparison's honesty needs the boundary cases, and the text line still wins two. The debug session on the developer's machine, the console output that a human reads while stepping through code, the structured event's JSON is the noise there and the printf is the tool, and the discipline that ships production logging should not slow the local debug. The emergency, the store is unreachable and the engineer is tailing the container output live, the human readable line is the format the emergency wants, and the pipeline's shape, structured to the collector, readable at the console, is the both-and that the good structured formats achieve, the single line event whose fields a human can still read in a tail.
The migration from the text estate is the strangler pattern the legacy article taught, the collector parses the old service's text into the structured contract at the boundary, the new services write structured from their first deploy, and the query surface unifies before the logging calls do, the event vocabulary mapped onto the legacy phrases in the collector config, and the archaeology retires one service at a time.
The working rules for the structured estate
The practice compresses into the rules that fit the wall, and the list is short:
- One line one event, with a named event type from the team's owned vocabulary;
- Two field classes, bounded dimensions for the queries, context for the drill down, and never the unbounded index;
- Propagate the trace identifier into every event, the correlation that the queries and the traces both consume;
- Log no personal data, the review item that the compliance audit will otherwise name;
- Ship through the collector, enrich at the boundary, and version the pipeline config with the code;
- Price the store by the ingestion and the query surface, and the retention by the compliance calendar.
The log line is the system telling itself its own story, and the structured line is the story written to be queried, the named events and the propagated identifiers that turn the incident's first minute from archaeology into a filter, and the year's logs from an archive into a dataset. The grep is not dead, it is demoted, the local debug tool it always was, while the production questions move to the contract the fields define.