Facebook Launches New Open-Source OS Monitoring Tool

Today Facebook is shaking up the OS monitoring ecosystem with the release of a new open-source project around operating system analytics and monitoring for multiple platforms including Ubuntu, CentOS, and Mac OSX. Called "osquery," the project is based on a modular framework shared by Facebook, to offer the security community a more affordable means of low-level operating system monitoring. In addition to asking for community involvement to build out components for the osquery framework, Facebook hopes to engage researchers to help harden the code already built by including it in its corporate bug bounty program.

"After talking with several external companies, it became clear to us that maintaining insight into the low-level behavior of operating systems is not a problem which is unique to Facebook," says Mike Arpaia, a software engineer at Facebook, explaining the impetus for sharing the project. He says this follows several months of sharing osquery with several external companies, which have used it and offered feedback for a wider open-source release.

The osquery framework is designed to improve operating system troubleshooting and monitoring by representing abstract operating system concepts as database tables that can be queried.

"This design allows you to write SQL-based queries efficiently and easily to explore operating systems," Arpaia says. "With osquery, SQL tables represent the current state of operating system attributes, such as running processes, loaded kernel modules, and open network connections."

Two of the defining features of the project as it stands are its interactive query console and its high-performance host monitoring daemon. The query console, osqueryi, offers up an SQL interface for exploring an operating system in order to diagnose systems operations problems and troubleshoot performance issues, Arpaia says. Meanwhile, the monitoring daemon, osqueryd, gives users the power to schedule queries across their infrastructures.

"The daemon takes care of aggregating the query results over time and generates logs, which indicate state changes in your infrastructure," Arpaia says. "You can use this to maintain insight into the security, performance, configuration, and state of your entire infrastructure."

This logging can also integrate into various log aggregation and log management platforms through osquery's plugin architecture. The framework's modular codebase is designed so that components like osqueryi and osqueryd can be "easily strung together" via a number of documented public APIs.

"Osquery was built so that every environment-specific aspect of the toolchain can be hot-swapped at run-time with custom plugins. Use these interfaces to deeply integrate osquery into your infrastructure if one of the several existing plugins doesn't suit your needs."

As for the bug bounty, Arpaia also announced today that Facebook is offering a minimum price tag of $2,500 for responsibly disclosed vulnerabilities in osquery core code, with rewards scaling upward based on severity. Some of the classes of bugs eligible for a bounty include privilege escalation and remote code execution.

Arpaia encourages researchers to take particular care poring over osqueryd, because it has the largest attack surface of all the components. He told researchers that the easiest way to find vulnerabilities would be to look at the SQL tables that osqueryd depends on to schedule queries.

"Many tables, like the 'apps' table and the 'launchd' table on OS X, do quite a bit of file parsing. If I were trying to find a vulnerability in osquery, I would look at those tables first. For example, the plist parsing code can be found at osquery/filesystem/darwin/plist.mm. Perhaps a specially formatted property list file could be created that causes unexpected behavior."