NGINX and Netflix Contribute New sendfile(2) to FreeBSD

Open source is the driving force for more and more innovation in technology and business, and open source is the new standard for application development, as described in our blog post, The Three Waves of Open Source Development.

NGINX is one of the most significant open source projects, powering nearly half of the world’s busiest websites. We’re proud of the many significant contributions from our team, both at work and individually. Open source technology and contribution are core to our values as a company.

Now NGINX has teamed up with our partner Netflix, another driving force in open source, to offer a new implementation of the sendfile(2) system call, for FreeBSD only, which further speeds up TCP data transfers.

sendfile runs fast because it avoids copying file data into a buffer before it’s sent. The new version of sendfile further speeds up and simplifies large data transfers by supporting asynchronous I/O.

The new sendfile is a product of the ongoing development partnership between NGINX and Netflix, and was released in tandem with Netflix’ recent service expansion to nearly 200 countries. Netflix is using it in production to send multiple tens of gigabytes per second.

The new implementation of sendfile adds two new flags and redefines an existing flag:

• SF_NOCACHE tells the kernel that data is not to be cached after it’s sent.
• SF_READAHEAD is a macro that allows you to specify the read-ahead size in pages.
• SF_NODISKIO no longer blocks, and returns EBUSY in the rare
  situations where the file is temporarily busy. In this case, you can retry the operation in a moment.

The new sendfile is a drop-in replacement for the previous one. You don’t need to make any changes to your applications. However, you can now expect significantly better performance in cases where the old sendfile blocked on disk I/O, and you can add the SF_NOCACHE flag and the SF_READAHEAD macro, with a page count, to modify the action of sendfile.

For more information, please see these related links:

• Basic operation of sendfile
• SVN commit message for new sendfile
• Use of sendfile by Netflix
• Thread pool use with sendfile
• Reference documentation for sendfile and sendfile_max_chunk (ignore the note about SF_NODISKIO)

Note: sendfile bypasses user space, so filters that change content, such as gzip, do not normally work in combination with sendfile. To avoid bypassing filters, NGINX automatically disables sendfile when a configuration context includes both sendfile and one or more directives that activate a filter which modifies content.

Image appearing for this post on the NGINX, Inc. blog home page is courtesy of NASA.