Comments on: Server utilization: Joel on queuing http://www.johndcook.com/blog/2009/01/30/server-utilization-joel-on-queuing/ The blog of John D. Cook Sat, 11 Feb 2012 22:42:11 -0500 http://wordpress.org/?v=2.8.4 hourly 1 By: John http://www.johndcook.com/blog/2009/01/30/server-utilization-joel-on-queuing/comment-page-1/#comment-33906 John Wed, 03 Mar 2010 14:09:55 +0000 http://www.johndcook.com/blog/?p=1395#comment-33906 The waiting time does not become negative. The horizontal axis is drawn at waiting time = 2. The waiting time does not become negative. The horizontal axis is drawn at waiting time = 2.

]]> By: Dilemma http://www.johndcook.com/blog/2009/01/30/server-utilization-joel-on-queuing/comment-page-1/#comment-33905 Dilemma Wed, 03 Mar 2010 14:06:33 +0000 http://www.johndcook.com/blog/?p=1395#comment-33905 Why the waiting time becomes with less than 0.5 utilization negative? Why the waiting time becomes with less than 0.5 utilization negative?

]]> By: Neil Gunther http://www.johndcook.com/blog/2009/01/30/server-utilization-joel-on-queuing/comment-page-1/#comment-16930 Neil Gunther Mon, 04 May 2009 15:38:23 +0000 http://www.johndcook.com/blog/?p=1395#comment-16930 Cool! Now, I can show you the correct *rules-of-thumb* (ROT), which Software Joel did not actually state; he only provides a vague impression. Looking at the stretch factor F=R/S, then the first ROT is: (A) if the server or cashier is idle, i.e., no traffic coming to her checkout, then r=0 and therefore F=1 or R=S. In other words, your total residence time (R) when you arrive at the checkout is just the time spent ringing up your groceries, because nobody was ahead of you. ROT (B) Suppose the cashier is idle only 50% of the time (moderate traffic). Then, r=1/2 and F=2. In other words, when the cashier becomes 50% busy, your expected residence time climbs from 1 service period (S) to 2 service periods, i.e., R=2S. You can see this (not so clearly) in your plot above; the curve cuts the x-axis at r=1/2 and the corresponding y-value is 2---not shown. ROT (C) If the cashier becomes 75% busy r=3/4, F=4 or R=4S (heavy traffic). This case is quite clear in your plot. In other words, your residence time doubled again, even though the server only became 50% busier than case (B). This is due to the intrinsic *nonlinearity* of the queue (the hyperbolic function) and that's what makes queueing theory rather unintuitive (even for experts). Cool! Now, I can show you the correct *rules-of-thumb* (ROT), which Software Joel did not actually state; he only provides a vague impression. Looking at the stretch factor F=R/S, then the first ROT is: (A) if the server or cashier is idle, i.e., no traffic coming to her checkout, then r=0 and therefore F=1 or R=S. In other words, your total residence time (R) when you arrive at the checkout is just the time spent ringing up your groceries, because nobody was ahead of you. ROT (B) Suppose the cashier is idle only 50% of the time (moderate traffic). Then, r=1/2 and F=2. In other words, when the cashier becomes 50% busy, your expected residence time climbs from 1 service period (S) to 2 service periods, i.e., R=2S. You can see this (not so clearly) in your plot above; the curve cuts the x-axis at r=1/2 and the corresponding y-value is 2—not shown. ROT (C) If the cashier becomes 75% busy r=3/4, F=4 or R=4S (heavy traffic). This case is quite clear in your plot. In other words, your residence time doubled again, even though the server only became 50% busier than case (B). This is due to the intrinsic *nonlinearity* of the queue (the hyperbolic function) and that’s what makes queueing theory rather unintuitive (even for experts).

]]> By: Neil Gunther http://www.johndcook.com/blog/2009/01/30/server-utilization-joel-on-queuing/comment-page-1/#comment-16908 Neil Gunther Mon, 04 May 2009 07:09:12 +0000 http://www.johndcook.com/blog/?p=1395#comment-16908 In your post, you appeal to a queueing theory formula (for M/M/1 waiting time), which kinda comes out of nowhere. However, there's a genuine intuition behind the shape of the curve---beyond a rule-of-thumb. It goes like this. You're in a grocery store and it will take time S to ring up your groceries. But when you arrive at the checkout, there are N people ahead of you. Each one them also takes time S (on average) to check out their groceries. The total time you now expect to spend at the checkout is R = W + S, where W=NS is your average waiting time and S is your own service time once you get to the cashier. Clearly, W<R, so it follows that W must be some fraction of the total time R or W = R.r where r is a real number in the range 0 to 1. Now, we have: R = R.r + S and rearranging produces R=S/(1-r). This is the "residence time," not the "waiting time" (slight bug in your argument). It should be clear that my 'r' is your Greek letter, rho. Moreover, writing F=R/S=1/(1-r), which is the hyperbolic function that you plotted. This function also has a name: the stretch factor. It's also possible to relate F to the Unix/Linux <a href="http://perfdynamics.blogspot.com/search?q=stretch" rel="nofollow">load average</a>, which is also used a gauge of server performance. In your post, you appeal to a queueing theory formula (for M/M/1 waiting time), which kinda comes out of nowhere. However, there’s a genuine intuition behind the shape of the curve—beyond a rule-of-thumb. It goes like this. You’re in a grocery store and it will take time S to ring up your groceries. But when you arrive at the checkout, there are N people ahead of you. Each one them also takes time S (on average) to check out their groceries. The total time you now expect to spend at the checkout is R = W + S, where W=NS is your average waiting time and S is your own service time once you get to the cashier. Clearly, W<R, so it follows that W must be some fraction of the total time R or W = R.r where r is a real number in the range 0 to 1. Now, we have: R = R.r + S and rearranging produces R=S/(1-r). This is the “residence time,” not the “waiting time” (slight bug in your argument). It should be clear that my ‘r’ is your Greek letter, rho. Moreover, writing F=R/S=1/(1-r), which is the hyperbolic function that you plotted. This function also has a name: the stretch factor. It’s also possible to relate F to the Unix/Linux load average, which is also used a gauge of server performance.

]]> By: Gene http://www.johndcook.com/blog/2009/01/30/server-utilization-joel-on-queuing/comment-page-1/#comment-12656 Gene Sun, 01 Feb 2009 14:49:46 +0000 http://www.johndcook.com/blog/?p=1395#comment-12656 In a restaurant, when wait times grow, a good manager will routinely visit clients' tables, perhaps dropping off a free drink or extra hors d'oeuvres, and generally lowering the tension level. Many patrons appreciate the attempt and stress levels go down. The service isn't any better, but we feel better. Now, picture an Oracle server, executing on a multi-processor Linux box, trying the same approach: "I'm sorry, the queue is full and your process is waiting for execution. Would you care for an additional Quote-Of-The-Day? Or perhaps I can spawn another copy of xterm while you wait?" It's all about the context of the waiting period, isn't it? In a restaurant, when wait times grow, a good manager will routinely visit clients’ tables, perhaps dropping off a free drink or extra hors d’oeuvres, and generally lowering the tension level. Many patrons appreciate the attempt and stress levels go down. The service isn’t any better, but we feel better.

Now, picture an Oracle server, executing on a multi-processor Linux box, trying the same approach: “I’m sorry, the queue is full and your process is waiting for execution. Would you care for an additional Quote-Of-The-Day? Or perhaps I can spawn another copy of xterm while you wait?”

It’s all about the context of the waiting period, isn’t it?

]]>