OpenAI's Codex Can Now Use Your Mac Even When It's Locked

OpenAI has significantly enhanced its Codex desktop application for macOS with a new "Computer Use" feature, allowing the AI coding agent to interact with and operate applications on a user's Mac even when the device is locked and its screen is off. This innovative update addresses a long-standing need for developers to manage and execute AI-driven coding tasks remotely, providing greater flexibility and continuity in their workflows. The core functionality revolves around enabling Codex to receive tasks from a user's mobile device and then autonomously perform actions on the locked Mac. This means developers no longer need to keep their computers unlocked or actively monitored for Codex to complete its assignments. For the feature to work, users must install the "Computer Use" plugin and grant it necessary macOS permissions, including Screen Recording and Accessibility. Once configured, Codex gains the ability to navigate through windows, type, interact with menus, and manage the clipboard within applications that have been explicitly allowed by the user. OpenAI highlights the utility of this feature for tasks that are difficult to automate via command-line tools. Examples include reproducing bugs specific to graphical user interfaces, modifying application settings, or running complex flows within desktop applications that Codex is actively helping to build. To maintain security and user control, Codex will prompt for permission before operating any new application. For frequently used applications, users have the option to mark them as "Always allow" to streamline the process further. Security is a key consideration for this new capability. OpenAI states that the mobile system utilizes a "secure relay layer" to ensure that the computer is not directly exposed to the open internet. Additionally, any local input on the Mac during a remote session triggers an alert, revoking the unlock state and reactivating the lock screen, thereby preventing unauthorized access. The feature also operates within a restricted sandbox, preventing Codex from controlling Terminal applications, interfering with itself, or executing `sudo` commands or system prompts. This secure environment is facilitated by an Apple-authorized low-level accessibility plugin into macOS. This "Computer Use" feature is part of a broader set of recent enhancements to Codex. Other notable additions include "Appshots," which allows users to capture a screenshot and accompanying text from a Mac app window directly into a Codex thread using a simple Command-Command shortcut. Another update is the new `/goal` mode, designed to enable an agent to persistently work towards a specific milestone over extended periods, potentially spanning hours or even days. It is important to note that, at its launch, this feature is not available in the European Economic Area, the UK, and Switzerland.