POSIX Threads Don’t Scale Past 100K Concurrent Web Service Requests (Part 1/2)

Understanding the graph below requires us to take a slight detour and explain the testing environment, the testing software, the various ways that we could process web services, the types of tests being run and the software that is running the test. This explanation assumes that you know the difference between an operating system-level process, a thread and a light-weight thread (which we call a fiber).

The Testing Environment

For this particular test, we:

  • wanted to see how processing throughput for web services would be affected by different processing models.
  • did not want to confound matters by running network-based tests as network delay would trump scheduling delay by several orders of magnitude. We focused on utilizing only the CPU and RAM in the machine.
  • wanted to see how well we utilized some of the newer dual-core and quad-core commodity PC processors that are hitting the market.
  • wanted to see how our in-house MODEST Web Services Processor would handle the processing load.

The testing hardware consisted of a single, quad-core machine with the following hardware:

  • AMD Phenom 9150E 1.8GHz Socket AM2+ 65W Quad-Core Processor
  • Gigabyte GA-MA78GM-S2H AM2+/AM2 AMD 780G HDMI Micro ATX AMD Motherboard
  • G.SKILL 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel RAM

The testing software consisted of the following:

  • Operating System: Kubuntu Linux
  • Kernel: 32-bit Linux kernel 2.6.24-19-generic #1 SMP. Note that the kernel did not have PAE support, so the total RAM available was only 310567KB (3.1GB)
  • Test Software: Digital Bazaar MODEST Web Services Processor
  • Tests: Each work unit consists of 50 Encode/Decodes of a JSON object

The Testing Software

The Digital Bazaar MODEST Web Services Processor is a cross-platform (Windows, Linux and Mac OS X) C++ application framework. It supports various web service features such as a HTTP micro-server, dynamic objects, inter-fiber communication, a module-based architecture, plug-in support for extensibility, a tiny server footprint (less than 2MB in most cases), JSON Remote Procedure Call API and various other features that are helpful when creating scalable web services.

The MODEST Web Services Processor also provides a micro-kernel and fiber scheduler coupled with a message-passing architecture that allows you to mix and match different processing models to provide web services to your customers. It contains the ability to mix and match pure POSIX threads, thread pools, operations and fibers in different configurations to serve web service requests. We will be open sourcing the technology in the next year – this is the first time that we’ve mentioned the MODEST Web Services Processor publicly.

The Processing Models

We used MODEST to test out three different approaches for handling and processing web service requests:

  • Pure POSIX threads – one thread per request (500 threads)
  • MODEST Operations (500 operations, running on 4 threads)
  • MODEST Fibers (500 fibers, running on 4 operations on 4 threads)

The first approach, pure POSIX threads, starts a new thread for every web service request. Each thread will concurrently encode and decode a fairly complex and verbose JSON object 50 times.

The second approach, MODEST Operations with 4 threads, utilizes a thread pool with one thread per CPU core and starts a new Operation for every web service request. Each operation will encode and decode a fairly complex and verbose JSON object 50 times in one shot.

The third approach, MODEST Fibers (4 threads), utilizes a thread pool with one thread per CPU core. There is one operation per CPU core, and one fiber per web service request. Each fiber will concurrently encode and decode a fairly complex and verbose JSON object 50 times, performing one encode/decode for every time it is scheduled to run.

The experiment that we wanted to run was to push each approach as hard as we could and discover the difference between each performance profile. Keep in mind that the important part is not the subtleties of how the JSON object is encoded and decoded as the work that needs to be performed is the exact same with each approach. The variable in the experiment is the processing model used to schedule 500 concurrent web service requests.

When we ran each processing model through the test suite and were surprised by what we found…

CONTACT US

We're not around right now. But you can send us an email and we'll get back to you, asap.

Sending

© 2019 Digital Bazaar, Inc. All rights reserved.

Log in with your credentials

Forgot your details?