Container base image playbook: the first 72 hours when a flaw is in the image your services build on
- 0 to 4 hours: 4 steps
- 4 to 24 hours: 4 steps
- 24 to 72 hours: 2 steps
- Hours elapsed: the bar under the cells
Checklist to print
0 to 4 hours4 steps, due by hour 4
- 1
Platform engineering, hand-off
List every image running in the cluster, with init containers, and keep the list as your scope. kubectl get pods -A -o jsonpath='{range .items[*]}{range .spec.containers[*]}{.image}{"\n"}{end}{range .spec.initContainers[*]}{.image}{"\n"}{end}{end}' | sort -u > running-images.txt. For hosts that run plain Docker, docker ps --format '{{.Image}}' | sort -u. Take the list from the running system, not from the registry: the registry also holds images nobody runs.
Done at
- 2
Platform engineering, hand-off
Find which images contain the affected package, without running them. For Alpine the package database is at lib/apk/db/installed, so for one image: crane export "$img" - | tar -xOf - lib/apk/db/installed | grep -A1 -x 'P:openssh-server'. The line after P: is V:, the version. Loop over running-images.txt and over the registry: crane catalog registry.example.com, then crane ls <repo>. Fixed openssh-server versions per Alpine branch: edge 10.6_p2-r0; 3.24 10.3_p1-r2; 3.23 10.2_p1-r1; 3.22 10.0_p1-r11; 3.21 9.9_p2-r1; 3.20 9.7_p1-r6; 3.19 9.6_p1-r3; 3.18 9.3_p2-r4; 3.17 9.1_p1-r7; 3.16 9.0_p1-r6. Compare the package version, not the banner printed by ssh -V. A scanner gives the same answer: syft "$img" -o json | jq -r '.artifacts[] | select(.name=="openssh-server") | .version'. Debian-based images use the same idea with /var/lib/dpkg/status.
Done at
- 3
Platform engineering, hand-off
Find the images built on a vulnerable base even when their Dockerfile says nothing useful. Layer check: save the base and the candidate configs with crane config "$base" > base.json and crane config "$app" > app.json, then jq -n --slurpfile b base.json --slurpfile a app.json '($a[0].rootfs.diff_ids[:($b[0].rootfs.diff_ids|length)]) == $b[0].rootfs.diff_ids' prints true when the candidate starts with every layer of the base. Source check: grep -rEn '^FROM .*alpine' --include='Dockerfile*' . across your repositories, and note every FROM that names one of your own images, since those are intermediates that also need a rebuild.
Done at
- 4
Security lead, hand-off
Cut the list to images that run the vulnerable daemon and are reachable. Does it start sshd: crane config "$img" | jq '.config.Entrypoint, .config.Cmd, .config.ExposedPorts' and look for sshd or 22/tcp. Is the exposed condition true in a running container: kubectl exec <pod> -- sshd -T | grep -iE 'passwordauthentication|usepam|permitrootlogin' (sshd -T needs host keys, so run it in the container started normally), then kubectl exec <pod> -- awk -F: '$2=="*"{print $1}' /etc/shadow for accounts with no password, and awk -F: '$2 ~ /^\$[56]\$/{print $1}' /etc/shadow for accounts with a SHA-256 or SHA-512 hash. Both lists non-empty with PasswordAuthentication yes and the port reachable puts the container in scope. Containers that do not run sshd are not affected, according to Alpine.
Done at
- 1
4 to 24 hours4 steps, due by hour 24
- 5
Platform engineering, hand-off
For in-scope containers that cannot be replaced within hours, set PasswordAuthentication no. Ship it as a mounted sshd_config or a drop-in in /etc/ssh/sshd_config.d/, check it in the container with sshd -t, and confirm with sshd -T | grep -i passwordauthentication after a restart of sshd. Look in Match blocks too. Before you roll it out, prove key login still works from a second session, because the change locks out every password user. MaxAuthTries 1 is a third-party stopgap that Medcrypt reports keeps password login while blocking its reproduction; treat it as a bridge only.
Done at
- 6
Platform engineering, hand-off
Rebuild in layer order: base, then intermediates, then applications. In the base Dockerfile add RUN apk upgrade -U -a, which is the update command Alpine gives, and build with docker build --pull --no-cache so nothing comes from cache. Alpine point releases 3.21.10, 3.22.8, 3.23.8 and 3.24.4 contain the fixed package. Branches 3.16 to 3.20 got no new point release, so a tag like alpine:3.20 does not help and the upgrade line is required. Change any FROM alpine@sha256:... pin to the new digest. Verify each build: docker run --rm --entrypoint apk <new-image> info -v openssh-server, and compare it with the fixed version for the branch in step 2.
Done at
- 7
Application owner, hand-off
Run the acceptance tests on the rebuilt image before rollout. Key login must work: ssh -o PreferredAuthentications=publickey user@host true. Password login must be refused on images that keep PasswordAuthentication no: ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password user@host true must fail. Run the service's own health checks, and check that anything that logged in over SSH with a password, such as a deploy script or a backup job, has been moved to keys.
Done at
- 8
Platform engineering, hand-off
Preserve evidence, then replace. For each in-scope container reachable from the internet, save the filesystem before it is deleted: docker export <container> > ctr-<name>.tar, or kubectl cp <pod>:/var/log ./evidence and the same for /etc. Roll out the new image: kubectl set image deployment/<name> <container>=<new-digest> and kubectl rollout status deployment/<name>. Then prove nothing runs the old image by digest, not tag: kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name} {range .status.containerStatuses[*]}{.imageID} {end}{"\n"}{end}' | grep -F <old-digest>. An empty result closes the step.
Done at
- 5
24 to 72 hours2 steps, due by hour 72
- 9
Platform engineering, hand-off
Block the old tag and digest so no deploy can bring them back. Admission policy, Kubernetes 1.30 or later, as a ValidatingAdmissionPolicy with validation expression !object.spec.containers.exists(c, c.image.contains('myapp:1.4.2')) and the message 'image has CVE-2026-123158', with matchConstraints on pods, then a ValidatingAdmissionPolicyBinding with validationActions: [Deny]. Match the digest as well as the tag. In the registry remove or retag the old tag, for example aws ecr batch-delete-image --repository-name myapp --image-ids imageTag=1.4.2, or set the repository to immutable tags and publish the fixed image under a new tag. Cost of deleting: you lose rollback to that image, so keep a tested fixed rollback target first.
Done at
- 10
Security lead, hand-off
Hunt for logins that already happened, on every in-scope container that was exposed. Container logs: kubectl logs <pod> | grep 'Accepted password for', and look for accounts that have no password. Check the archived filesystems and live containers for the traces GreyNoise reports, such as /etc/init.d/tac-agent and a tac[.]vpnla[.]org entry in /etc/hosts. If you cannot rule out a login, Alpine says to treat the machine as compromised: rotate every secret mounted in the container and review the cloud roles it held.
Done at
- 9
A base image makes one flaw into hundreds. Every service that starts from the same FROM line carries the vulnerable package, and so does every intermediate image another team built from it. The flaw is exploited wherever the package runs, so the first question is not 'which images contain it' but 'which images run it, and where'. In CVE-2026-123158 the package was sshd in Alpine's openssh-server. Most Alpine images never start sshd, and those are not exposed. An image that does start it, with password login on, lets an attacker log in without a password.
Patching is the hard part, because the fix is not applied to anything that runs. A fixed package in the distribution repository changes nothing until an image is rebuilt, pushed under a new tag or digest, and the old containers are replaced. Pinned digests, cached build layers and images built from your own base images all keep the old package alive. The steps below follow that chain: find, narrow, mitigate, rebuild in order, replace, block, and check.
When no fix exists yet
No fixed image can mean three things. The distribution has a package but no new point release (Alpine 3.16 to 3.20), so the fix reaches you through apk upgrade in the build and not through the tag. The image is on a release that no longer gets updates, or its vendor ships no fix. Or an appliance image bundles the package and you cannot rebuild. Your options are in order of cost. Set PasswordAuthentication no. Remove the daemon the image does not need: containers rarely need sshd, kubectl exec replaces it, and apk del openssh-server makes the flaw unreachable. Build from the upstream fix: Medcrypt reports the OpenSSH fix commit applies cleanly to OpenSSH 7.3p1 and later, at the cost of a package you now maintain.
Take the container offline, or block port 22 at the network policy or security group, when all three are true: sshd is reachable from untrusted networks, an account has no password while another has a SHA-256 or SHA-512 hash, and the password setting cannot change. OpenSSH's own wording for musl systems without PAM is to set PasswordAuthentication=no or disable SSH immediately. Disabling SSH cuts remote administration of that image, so make sure the cluster or console access works first.
CVE-2026-123158 worked through, hour by hour
The OpenSSH password bypass of 5 Nov 2026 is a base-image case because the affected package was Alpine's openssh-server, built without PAM against musl libc. The CVSS base score of 8.1 comes from the MITRE CVE record; NVD has no rating of its own. This table keeps the moments that matter for an image. The full dated sequence, from the fix commit to the exploit code, is in the Linux server SSH playbook.
| When | What happened | Source |
|---|---|---|
| 5 Nov 2026, 09:00 UTC (hour 0) | Published: the OpenSSH fix is public at 09:00 and 10.6p2 is released as an urgent security release at 09:01. Alpine publishes its advisory and fixed packages the same day, including the 3.21.10, 3.22.8, 3.23.8 and 3.24.4 point releases. Response: platform engineering writes running-images.txt from the cluster (step 1). | source |
| 5 Nov 2026, hours 0 to 4 | Response, no new publication: the apk database is read from each running image and compared with the fixed versions, base and intermediate images are traced by layer, and the sshd and PasswordAuthentication check cuts the list to containers that run the daemon (steps 2 to 4). Alpine says containers that do not run sshd are not affected. | source |
| 5 Nov to 6 Nov 2026, hours 4 to 24 | Response, no new publication: PasswordAuthentication no goes in where replacement takes longer than hours, the base is rebuilt with apk upgrade -U -a and --no-cache, then intermediates and applications. Exposed containers are exported, replaced by digest, and the imageID grep for the old digest returns nothing (steps 5 to 8). | source |
| 7 Nov 2026, 00:46 UTC (about hour 40) | Published: GreyNoise observes the first successful exploitation, on one of its own sensors. Intrusions with follow-up commands begin the same day. Response: the exposed containers were replaced about 16 hours earlier. | source |
| 8 Nov 2026, 09:00 UTC (hour 72) | Response, no new publication: the admission policy and registry block for the old tag and digest are in place, and the hunt for accepted passwords and the GreyNoise traces is under way (steps 9 and 10). | source |
| 9 Nov 2026 | Published: CISA adds the CVE to the KEV catalog, due 12 Nov 2026. | source |
The 3.16 to 3.20 branches are the case where the tag does not move: only the apk upgrade line in the build and the imageID check at step 8 show the fix landed. A team that started at the KEV entry on 9 Nov began 2 days after the first success on a GreyNoise sensor.
Related pages
Sources
- Alpine Linux security advisory, 5 Nov 2026, read 19 Nov 2026
- OpenSSH 10.6p2 release notes, read 19 Nov 2026
- OpenSSH fix commit, read 19 Nov 2026
- CISA KEV feed, catalog version 2026.11.18, read 19 Nov 2026
- GreyNoise event page and timeline, read 19 Nov 2026
- Researcher write-up by Ozgun Kultekin, read 19 Nov 2026
- Medcrypt reproduction and mitigation guidance, read 19 Nov 2026
- NVD record for CVE-2026-123158, read 19 Nov 2026
Last reviewed 19 Nov 2026.