Introduction to the Apple Endpoint Security Framework

Introduction to the Apple Endpoint Security Framework

An introduction to a game-changer for MacOS Security, and how to leverage it for Threat Hunting and Forensics

·

9 min read

Cover Illustration by ireneparamithaa

For years there are two camps of perception in MacOS security, those who think that Macs are impenetrable boxes by design and sysadmins who are constantly horrified by the lack of protections that MacOS has despite being a more locked-down platform than Windows. For a time these two camps coexist simply because for years MacOS was never an attractive platform to attack by threat actors due to its low adoption rate.

But as Apple's marketshare grows, especially for the enterprise market and particularly in tech firms, more and more complex malware are beginning to attack MacOS systems. The SmoothOperator 3CX supply chain compromise and Lockbit's foray into Mac ransomware are evidence that many threat actors are now moving toward Mac as a platform to attack.

For a time, Apple allowed third party code, especially security products, to run in kernel-land via kexts (Kernel Extensions), which if you come from the Hackintosh scene might bring parallels to how drivers work in Windows. I have written extensively how this approach in Windows didn't really worked out well, giving programmers the ability to run code inside the kernel (even trusted developers that are vetted through programs such as WHQL) sometimes can bring unintended consequences.

Apple's Endpoint Security Framework (ES) is a C API made by Apple as a solution for EDR/AV vendors to monitor OS telemetry events in the userspace similar to Windows ETW-TI. Since the implementation of the ES API, monitoring technologies via kext and the default Full Security level in iBoot have become largely outdated. This shift is evident in the XNU source code, where developers have introduced the __kpi_deprecated(_msg) macro, which generates a compile-time warnings, advising the use of EndpointSecurity instead.

A More Open Endpoint Security Standard

Similar to Microsoft's approach in Windows via ETW-TI, ES provides a way for EDR/AV systems to run in user-mode privileges while still retaining the security and protection they enjoy running in kernel space.

For your userspace apps to successfully register with the ES API you must be properly entitled with the com.apple.developer.endpoint-security.client entitlement, which requires an entitlement from Apple via the Apple Developer System Extensions Request Form.

This is kinda more similar to the requirement for Early Launch Anti-Malware (ELAM) drivers that must adhere to specific program requirements set by Microsoft, including being signed by the Windows Hardware Quality Lab (WHQL) and the necessity for antimalware vendors to be part of the Microsoft Virus Initiative (MVI) to participate in the ELAM program​​​​​​.

Of course, while Apple is selective on who can get access to the ES, it is still more open compared to the standards instilled by Microsoft with you needing to be an MVI vendor to be build an ELAM driver.

General Architecture Overview

There are three main components of ES :

  • libEndpointSecuritySystem.dylib

    This is a userland dynamic library for apple to link MacOS components to the ES API so they can emit events into EndpointSecurity.kext, which allow binaries to emit ES events around background task management, XProtect detection events, sudo invocation, TouchID usage, screensharing/recording activity, and OpenSSH logins

  • libEndpointSecurity.dylib

    This is userland dynamic library for developers working with ES that handles the access of several abilities reserved for EndpointSecurity.kext safely via API to third party applications, so when developers build with ES API signed by the required entitlement, they will link it against libEndpointSecurity.dylib. Doing this allows apps to subscribe to ES events, applying path muting, and authorizing system activity

  • endpointsecurityd

    This is a userland daemon that does validation to the system extensions that use ES by validating its privileges, enables the installation/uninstallation of the extension, registering the system extension as an early boot process (with NSEndpointSecurityEarlyBoot in info.plist, similar to ELAM on Windows), and the recording of analytics through the CoreAnalytics framework

  • EndpointSecurity.kext

    This is a kernel extension made by Apple that enables proxying requests from EF API system extensions that exist userspace through Apple's own kernel drivers, which enables more security and control instead of just letting applications themselves inline to the XNU kernel

ES events come from both the hooking the system calls of kernel-level events and through the libEndpointSecuritySystem.dylib dynamic library for system components which much higher levels of abstraction. Processes that do not have the com.apple.private.endpoint-security.submit entitlement are explicitly forbidden from giving ES events to EndpointSecurity.kext to protect the integrity of ES event logs.

ES Events for Threat Hunting

Building your own system extension and building a pipeline to display it might be trivial for some individuals, but the entitlement needed from Apple provides a barrier for many from using this API. A way to circumvent this is to create a system extension to query the ES API without the entitlement and running it with System Integrity Protection (SIP) disabled, but within a forensics or incident response scenario this might contaminate the endpoint in a breach scenario. Thus this is where Mac Monitor by Red Canary comes in.

Mac Monitor allows us to read ES events in an enriched way, and also gives us a more graphic user interface and a set of event filters that makes us able to read certain events. In a way, this is similar to a more beefed up version of Windows Process Monitor.

Searching for Programs with Malicious Activity

To demonstrate the value of ES for threat hunting, we can try it by detonating a sample in a VM. I'm gonna use the Calisto malware, accessible via the Objective See Foundation Mac Malware Collection. Calisto is an infostealer that masquerades as an installer for Intego's Mac Internet Security X9, an personal security product for MacOS.

Upon execution, we can open Mac Monitor to see of any suspicious events. We can see immediately that the installer is flagged almost instantly by ES.

Looking deeper, we can see that this is flagged as a ES_EVENT_TYPE_NOTIFY_EXEC which notifies ES that a process is executing an image, by itself its not supposed to be something suspicious but what makes it pops out is that the application is not signed at all. Considering this is supposedly a legitimate security product with hooks into MacOS, its unusual for it to have no codesigning whatsoever.

Digging into the event correlation tab, we can see several events correlated with the application in question. Since Calisto is an old malware from 2018, we can see the things that it does in writeups that people have made. Since System Integrity Protection is enabled on the VM, we can only see the events that correspond to the malware's infostealing functions.

From the ES_EVENT types we can find the following

Peeking in the directory will reveal that inside .calisto there is a zip file called KC.zip which contains a full duplicate of the MacOS Keychain folder

Searching for Malicious Code Execution

Another thing we can simulate and hunt through ES is suspicious code execution, and for this we can use Posix Atomic Test Harness to simulate the abuse of AppleScript, a scripting language for macOS that allows control over applications and parts of the operating system through inter-application messages called AppleEvents. It can be used to locate open windows, send keystrokes, and interact with almost any open application either locally or remotely.

Scripts can be executed in various ways: from the command line using osascript, within mach-O binaries using macOS Native APIs like NSAppleScript or OSAScript, or even as plain text shell scripts.

Atomic posixath includes tests for different execution methods of AppleScript, such as through the NSAppleScript API, the command-line utility osascript, shell scripts with a shebang (#!/usr/bin/osascript), and as part of applets or stay-open-scripts. These tests are crucial for understanding how AppleScript can be used (or abused) in various scenarios, including benign automation tasks or potentially malicious activities.

Upon execution, we found that the script works as expected and unimpeded.

But upon looking in Mac Monitor, we can see that the event is logged, and MacOs has flagged the "applet" process.

Reading into it, we can also read the complete process execution chain, which can help in threat detection to detect where the malicious execution originates and where its ending up in.

Blocking more esoteric attack methods are key to improving EDR products in MacOS, especially knowing how lackluster solutions like Gatekeeper and XProtect can be sometimes iffy against blocking threats. While enabling SIP and Gatekeeper can stop around 80% of attacks, there will always be those weirder attack chains that EDRs need more advanced telemetry sources to detect.

Other Smoking Guns in ES Logs

While the events above show entrypoints for you to start your threat hunting journey, there are other events that are of interest in a threat hunting perspective that provides clear "smoking guns" in your investigation. Within your investigation you can filter and target these specific event types for threat detection. This is gathered using testing from different C2 platforms such as SliverC2, CobaltStrike (via Geacon), and MacShellSwift.

  • ES_EVENT_TYPE_NOTIFY_FCNTL Records the manipulation of a file descriptor, most of the time to get the file flag indicating a process is trying to gain dynamic access.

  • ES_EVENT_TYPE_NOTIFY_READLINK Records symlink reading operations, and most of these operations involve reading the link to /etc/, so any READLINK operation that doesn't point from SYSTEM to /etc/ should be considered malicious.

  • ES_EVENT_TYPE_NOTIFY_MMAP Records memory mapping operations on the system where it's possible to identify weird mappings that may be indicative of malicious activity.

  • ES_EVENT_TYPE_NOTIFY_MPROTECT Records all memory protection events, which can be seen as anomalous when run by interesting processes such as memory protection assigned to /bin/sh.

  • ES_EVENT_TYPE_NOTIFY_IOKIT_OPEN Records all process that calls to open an IOKit device, which can be useful in detecting malicious code thats performing screen capture or webcam monitoring operations which require IOKits related to hardware graphics acceleration. These events are usually triggered in a stream until the operation is finished so if you see an application that is using this IOKit without you using your webcam or screensharing, there is a big chance that you're actually being monitored.

  • ES_EVENT_TYPE_NOTIFY_PTY_GRANT Records all processes that grants a pseudoterminal device to a user, which is used to create a communication channel between a controlling terminal and a slave terminal. This is usually used to generate interactive shell instances and is rarely a normal system operation, so it appearing should be a red flag.

  • ES_EVENT_TYPE_NOTIFY_SETMODE Records changes in file access permissions via utilities like chmod, which we can utilize to monitor for access changes for non-standard binaries (outside of system directories like /bin/bash or developer tools like Xcode) to indicate malicious activity.

Conclusion

ES provides a multifaceted approach to threat detection and response. It serves as a powerful tool for monitoring system events for signs of malicious activity, enabling registered clients to receive notifications of such events. This functionality is crucial for developing advanced EDR solutions that can effectively monitor and, when necessary, block system events to conform to security policies and protect against potential threats.

From a threat detection perspective, the ES framework is instrumental in macOS security. Its capabilities extend beyond mere notification of events; it allows for a proactive stance in security management. For example, with authorization events, ES enables applications to take pre-emptive actions against potentially harmful processes. This feature is critical in stopping threats before they materialize into actual breaches or system compromises.

The existence of tools like Red Canary Mac Monitor also shrinks the skill gap required to do threat analysis in MacOS, which I honestly think is about time. I'm too bored doing detection engineering in Windows and with ES and tools like Mac Monitor I think MacOS Detection Engineering has a bright future.