A new engineer opens the service map and asks, “What happens after this worker fails three times?” The diagram has an unlabeled queue, the runbook links to an old service name, and every experienced teammate can answer from memory. That difference is the test. Give a newcomer one bounded question before offering a walkthrough, record exactly where the documentation made them guess, and have an owner verify the answer against the system. GitLab's onboarding handbook offers a public example of treating onboarding material as something teams maintain. This guide turns that idea into a repeatable architecture-documentation check.
Direct answer: In a new engineer's first week, give them a time-boxed task: explain the system using only the current architecture map and its linked decisions, before anyone walks them through it. Record every point where they need to ask someone. Turn those gaps into small documentation fixes that a system owner verifies, and let the new hire review the result. Do not make them rewrite the documentation.
Run a cold read
Give the new engineer 60 to 90 minutes with the documented map and decision links, and six questions:
- What are the main components?
- Which team owns each one?
- How does a representative request or event move through the system?
- Where are the trust boundaries?
- Which parts are deliberately asynchronous, and why?
- Where would you start investigating a failure in checkout (or your equivalent)?
Their confusion is the data you need, so hold the walkthrough until afterwards.
Record gaps, not just errors
Log each moment they needed help, with a category:
| Gap type | Example | Usual fix |
|---|---|---|
| Unexplained name | "What is Hermes?" | Rename, or label with its function |
| Missing purpose | A queue with no reason | Link the decision behind it |
| Ambiguous edge | Sync or async? | Label the relationship |
| Missing component | Found in code, not on the map | Update the map |
| Wrong place | Setup question | Fix the setup guide, not the map |
Many gaps are not errors. The diagram may be accurate but rely on conventions only long-time staff know. Fixing those helps every cross-team reader too.
Give the newcomer a question with a checkable answer
For a first-week exercise, ask a new engineer to find the service that owns order retries and explain what happens after the last failed attempt. Give them the current map and its linked decisions, with a thirty-minute timebox and permission to record every gap. A useful result is not “they felt confused.” It is a list such as “the worker is unlabeled,” “the dead-letter queue has no owner,” and “the linked decision explains the old flow only.”
Use the Arialine map if that team already reviews changes in Slack. The new engineer can point at the exact board version, request a missing label in its thread and inspect the accepted correction. A system owner checks the answer against the implementation before the diagram is changed. Repeat the same task after the fix with another reader; if they can now trace the failure path without a walkthrough, the documentation has improved in a way the team can observe.
Let the newcomer propose, and an owner verify
After the cold read, pair the new engineer with the system owner. The newcomer proposes clearer labels, missing links, and questions. The owner checks technical correctness. This turns onboarding into a real contribution without making the least-informed person the author of record.
Keep the map from absorbing everything
Not every question belongs on the architecture map. Broken commands go in the setup guide. Missing business context goes in product docs. Improve the map only where the confusion is about system structure or reasoning.
Measure it across hires
Track three things for each new hire:
- time to a correct high-level explanation;
- number of private clarifications needed;
- questions that repeat from previous hires.
If every new hire asks why the same queue exists, that decision record needs work. The goal is not zero questions. It is faster, better questions.
Where Arialine fits
If your map is an Arialine board, the cold read works from real artifacts. The root message shows the current diagram, the interactive HTML file has pan, zoom, and a Decisions panel, and History links each change to its source message, subject to the new hire's Slack access. The newcomer can post questions in the board's thread. An owner can turn the answers into proposed changes, and each accepted fix becomes a version. The history then shows how onboarding improved the map over time.
A guided version of this exercise is in the engineering onboarding walkthrough and its printable checklist.
Limits of the test
This tests whether a newcomer can recover a specific architecture answer from the available documentation; it does not measure the new hire's ability or prove that the entire documentation set is complete. Access problems, unfamiliar terminology and missing local setup can distort the result, so record those separately. Use a real but non-sensitive task, do not expose production credentials, and treat a failed test as a system problem to fix rather than a performance judgment about the participant.
FAQ
Should new hires edit documentation on day one?
They should flag confusion immediately. Technical changes should be verified by an experienced owner.
Is onboarding documentation different from architecture documentation?
Yes. Onboarding explains how to become productive. Architecture explains the system's structure and decisions. Link them, but keep them separate.
What if the map is deliberately incomplete?
Say so on the map: state the scope and what is left out. Incomplete documentation is fine. Documentation that is incomplete without saying so confuses readers.
Sources
- GitLab Handbook: General onboarding — reviewed September 27, 2026
- Google SRE book: Accelerating SREs to on-call and beyond — reviewed September 27, 2026
Add to my Slack