Recording break-glass access to Linux servers

The name comes from the fire alarm behind a glass panel: you only use it in an emergency, and everyone knows when you have. In access control, it's a pre-arranged route into a system for when the usual login path is broken.

Microsoft's guidance lists the typical triggers: the identity provider is down, MFA devices or the MFA service are unreachable, or the last admin has left. On a Linux fleet the emergency route is usually one of three things. It might be a root SSH key kept offline, the cloud provider's serial console, or a direct SSH login that bypasses the gateway.

Access gateways record at the point where they relay traffic. Teleport records at its SSH service or its proxy, and its OpenSSH setup has each server trust only certificates its proxy issues. HashiCorp Boundary records on the worker that handles a session. AWS states that SSH tunnelled through Session Manager isn't logged.

A break-glass login goes around all of that by design, because it exists for when the gateway or SSO can't be used. So the session where someone has the most power, during the worst hour of the year, is the one with no recording.

shellz replaces the login shell on every account. Whoever logs in, by key, by password or at a console, starts inside the recorder. Physical and serial console logins are recorded as their own session type. Your org policy can switch console recording off, so check that it's on for servers where the console is your fallback.

Switching to root with su - stays inside the recording. shellz terminates any login it finds running without the recorder and raises an alert. Passwords typed at hidden prompts, such as the one su asks for, aren't captured.

Outages often cut network links too. When a server can't reach shellz, it buffers recordings and uploads them once the connection returns.

A recording policy you set in the dashboard decides whether new logins are allowed while the server is offline. Refusing them keeps every session accounted for. It can also lock you out of a server whose console is your only way back. Settle this per server group in advance, not during the incident.

If a host falls silent, shellz alerts you in about a minute. During a network outage, expect those alerts and read them as part of the incident timeline.

Microsoft recommends alerting on every use of an emergency account and holding a review afterwards. The review decides whether the use was a drill, a real emergency or misuse, then examines what the person actually did. NIST SP 800-53 control AC-6(9) asks for privileged functions to be logged for the same reason.

A recording turns that review from interviews into evidence. The walkthrough below is hypothetical.

• ›At 02:10 the SSO provider fails. The on-call engineer takes the root key from the safe and logs in to db-01 directly.
• ›Next morning, the reviewer filters sessions by db-01, user root and the outage window. One session comes back.
• ›They replay it at high speed and read the transcript. It shows a restarted replication process and one edited config file.
• ›The session's integrity badge reads verified, so no part is missing or modified. They export the transcript as text and attach it to the incident ticket.

Break-glass users have full control of the machine, so the recording has to survive them. If root changes shellz's configuration, the change is reverted within seconds and reported. A local attempt to uninstall shellz is refused and reported, because removal needs approval in the dashboard.

Recordings leave the server encrypted, in signed pieces chained to each other. After upload they're immutable, and a hole or edit in the chain flags the session as gapped or tampered. A root user can always mess with programs on their own server, and shellz makes that visible.

shellz doesn't store or rotate the emergency key, approve its use, block commands or end a session. Those jobs belong to a password vault or PAM product. The PAM vs session recording guide explains when a small team needs one. For gateway-specific gaps, see shellz vs Teleport, Boundary alternatives and shellz vs AWS Session Manager.

Covering 10 active devices or fewer costs nothing. Larger fleets go on Commercial at $10 monthly for each shellz user and $1 monthly per GB kept (plans).