In the previous chapter, you read about ordered arrays and binary trees. In particular, you have learned how to work with ordered arrays of pointers to insert, delete, and find individual items.
KeywordsComparison Function Array Element Random Array Variable Swap Pivot Element
In the previous chapter, you read about ordered arrays and binary trees. In particular, you learned how to work with ordered arrays of pointers to insert, delete, and find individual items.
In this chapter, you will learn methods to bring order to unordered arrays.
The simplest way to sort an array is to go through its elements one by one and move them to their correct positions. This is what you do when you sort the cards in your hand while playing a card game.
Listing 9-1 shows how easy it is to apply insertion sort to an array of pointers.
The for loop that starts in line 5 selects an array element at a time starting from the second one.
In line 6, you save the address of the selected element. Then, in line 8, you compare the selected element with the elements that precede it within the array as long as those elements have a larger key.
When you exit the while loop, j points to the first element found that has a key below that of element k. This means that you should insert element k before element j+1. When all elements preceding k have a lower key, it means that k is already where it should be. When that happens, the while loop is never entered and j remains set to k-1. That’s why, after incrementing j in line 9, you only need to execute lines 11 and 12 if j < k.
In line 11, you move “right” one place the elements from j to k-1 (thereby overwriting the kth position) and, in line 12, you write the address of the kth element in the jth position.
The advantage of using this sorting method is that it is simple and doesn’t use extra memory because it sorts the array in place. Its disadvantage is that it might have to make lots of comparisons and memory moves.
We have only considered arrays in which all keys/values are different. But if you need to sort arrays in which the same key can appear more than once, insertion sort has another feature that you might find useful: it doesn’t swap around elements that have the same key. In other words, elements with identical keys remain in the order they had before sorting.
Incidentally, if you swap the two arguments of the comparison function invoked in line 8, the array is sorted in descending order.
Shell sort (named after Donald Shell, who first published the algorithm) is a variant of insertion sort. The idea is that you can move into place elements more quickly if you compare them with distant elements before applying the insertion sort as shown in the previous section. That is, when comparing an element with those that precede it within the array, you make more than one pass with decreasing distances between elements.
The tricky part is deciding how big the gaps should be. In fact, the problem has not yet been completely solved.
Listing 9-2 shows a variant of sort_insertion() that lets you choose the distance between elements. Then, to implement a Shell sort, you only need to execute the function repeatedly with decreasing distances.
As you can see, shell_step() is almost identical to sort_insertion(): in line 8 you replace
Then, to avoid accessing elements outside the lower array boundary when dist > 1, you change line 9 from
For example, you can implement sort_shell() as follows:
Incidentally, to avoid duplications in the code, you can replace the whole body of sort_insertion(), as shown in Listing 9-1, with shell_step(ar, cmp, 1);
It is interesting to play with the distances and see how they affect the number of comparisons.
Listing 9-3 shows a small program to explore that effect.
How Distances Affect the Number of Comparisons
The array dist defined and initialized in line 8 contains the distances used for the test. I initially included a handful of distances with gaps. This is why you see an array, although the values stored in it are a continuous sequence.
The for loop that starts in line 14 goes through all the distances. For each one of them you generate N_REP times a random array of pointers (line 17) and execute first a sort with a distance dist[k_dist] (line 20) and then an insertion sort (line 22). Note that you set the initial k_dist to -1 and only execute the Shell sort from the second iteration on, when k_dist becomes non-negative (line 19). In this way, the first iteration over k_dist produces a straight insertion sort.
The fill_random() function that you execute in line 17 is shown in Listing 9-4.
The for loop that starts in line 4 ensures that all elements of the integer array are used, while the do loop that starts in line 6 ensures that all elements of the pointer array are set. With the do loop, you keep selecting a random element of the pointer array until you find one that is available (i.e., that is NULL). It is a brute-force approach, but effective. With small arrays, it makes sense to look for free elements instead of designing an algorithm to keep track of the elements you have already set.
The comparison function you invoke in lines 20 and 22 of the program (Listing 9-3) is an updated version of the cmp_i() function you encountered in Chapter 8 (lines 9 to 13 of Listing 8-20). It is shown in Listing 9-5.
As you can see, the only difference is that it increments the global counter n_comp (defined in line 4 of Listing 9-3) but only when the debug flag defined in sort.h is set. By resetting the counter in the main program (line 18 of Listing 9-3) before each application of the Shell sort, you can print in line 23 the total number of comparisons performed in the sort.
Each point is calculated by averaging the N_REP repetitions for a particular distance, to make the curve more smooth. For example, the highest point, which corresponds to the straight insertion sort, is 2590. The minimum number of comparisons of the 100 repetitions was 2177 and the maximum was 3020.
As you can see from Figure 9-1, the best result (minimum average number of comparisons) was obtained by executing a Shell-sort step with a distance of 3 or 4 before executing the insertion sort (with total counts of 1550 and 1548, respectively).
In practical terms, you produce the curve with three steps (the Xs) by inserting immediately below line 20 of Listing 9-3 the line
and for four steps (the squares), you insert below line 20 of Listing 9-3 the following two lines:
Well, the bottom line is that, in our test, you obtain the best result by executing a total of three steps with 6, 3, and 1 distances, requiring an average of 1313 comparisons to sort the 100-element array. This is about half the number of comparisons you need with a single-step insertion sort (2590).
But how do you know how many steps and with what distances you obtain the best results with a particular set? Donald Knuth, one of the computer luminaries, recommends a method developed by Robert Sedgewick to calculate the number of steps and the distances. I couldn’t resist trying it out in the case of this example, although it might be argued that it is not very practical.
The first step to apply the method is to calculate a series of numbers hs for s = 1, 2, ...:
hs = 9 * 2 s - 9 * 2 s/2 + 1 if s is even
hs = 8 * 2 s - 6 * 2 (s+1)/2 + 1 if s is odd
If you do the calculations, the series h0, h1, h2, ... turns out to be 1, 5, 19, 41, 109, 209, ...
You stop calculating new elements when the triple of the element exceeds N, the size of the array you need to sort. In this case, N is 100. Therefore, only 1, 5, and 19 make the cut (because 3 * 41 already exceeds 100).
This means that according to Sedgewick, to sort 100 elements you should apply a distance of 19, then of 5, and finally of 1. I “patched up” the program of Listing 9-3 and made N_REP sorts with distances of 19, 5, and 1. The average number of comparisons was 1516, which is 15% higher than what I had obtained with 6, 3, and 1.
Well, it is not much higher, and the number of steps is the same...
As much as I like statistics and plots, it’s time to move on!
The bubble sort derives its name from the fact that air bubbles in a liquid raise to the surface. I am somewhat sentimentally attached to bubble sort because it is the first sorting algorithm I implemented in my programming career, decades ago. After implementing it, we shall see how it compares with the Shell-sort in terms of number of comparisons.
The idea of a bubble sort is simple: you go though the array and compare all its elements in pairs; if the second one should come before the first one, you swap them. When you are through, you do it again, and keep doing it until you no longer need to swap.
Like the insertion and Shell sorts, the bubble sort is done in place.
Check it out in Listing 9-6.
Pretty simple: you compare each element with the following one (line 11) and swap them if they are in the wrong order (lines 12 and 13). When you go through all the pairs without having to make a single swap, you are done.
I tested it with the code shown in Listing 9-7.
To my disappointment, it printed out a very high average number of comparisons: 8789. Several times higher than the best results of the Shell sort. But this result teaches us something: if you pick the first algorithm that does the job (as I did with the bubble sort long ago), you risk wasting resources. It always pays to check out a few options before settling on one.
Quicksort is binary and recursive: choose a pivot element; move the elements less than the pivot before the pivot and those greater than the pivot after the pivot; repeat the same process on the two sides of the pivot; keep going until the sides contain single elements, by which time you are done.
But it is easier said than done. First of all, how do you choose the initial pivot element and the pivots of the smaller portions of the array? And also, in most cases, you will need to accommodate more elements on one of the two sides of the pivot; what do you do then? And finally, in which order do you compare the elements with the pivot? Actually, does it matter at all?
Let’s start with a simple and easy example to familiarize ourselves with the idea. Suppose that your initial, unsorted array, is like this:
To make it simple, let’s assume that, with a stroke of luck (!), you choose the middle element as the pivot. That the middle element is also the median element (i.e., the element with equal numbers of lesser and greater elements) is unlikely, but for this first example, I want to keep things as simple as possible.
Starting from the beginning of the array, you look for an element whose key is greater than the pivot’s. This is E, in position 1. Similarly, starting from the end of the array, you look for a key that is less than the pivot’s. This is C, in position 6. Once you have both, you swap them, and the array becomes as follows:
You keep going and discover that G in position 2 is greater than the pivot, while B, in position 5 is less. You swap them and end up with the array
When you reach the pivot from both sides, you know that all the elements with lesser keys are on its left and all elements with greater keys are on its right. Then, you split the array in two and repeat the pivoting and swapping on the two halves:
Element 3, which contains D, is in bold to highlight that it is in its final position. It couldn’t be otherwise because we know that its key is greater than those of the elements on its left and less than those of the elements on its right.
Looking at the first half, you choose the middle element (C, in position 1) as the pivot. This time, scanning the elements from the beginning of the array section, you reach the pivot without finding any key that exceeds C. But when you scan the array section from the right, you find that B is less than C and should be moved to the left of the pivot. Unfortunately, there is nothing to swap it with. Therefore, you shift the pivot and all following elements to the right to make space for B.
You have the same problem with the section on the right side of the initial pivot. This time, after choosing once more the middle element as the pivot (i.e., G in position 5), you find that E should go to the left of G, but there is no element preceding G that you can swap it with. To resolve the issue, you make space for E by shifting to the right the pivot and all elements on its right (in this case, none).
After doing the partitioning on the two sides of the initial pivot, you have the following array:
The pivots of the two sides (C and G) are highlighted in bold to indicate that they are in their final places.
You know what’s going to happen next. You apply the partitioning to left and right sides of C and G. In the example, there are no elements to be partitioned on the right of either C or G.
To do the remaining partitioning, you choose A and F as pivots, discover that nothing needs to be moved around A and that E is to be moved to the left of F. Once you move E, you are done.
Listing 9-8 shows the sort_quick() function.
All sort_quick() does is check that the array contains at least two elements and invoke quick_step(), where you do the actual work (see Listing 9-9).
As it turns out, also quick_step() doesn’t do much processing: it invokes partition() to determine a pivot elements and split the array around it. As partition() moves the elements with keys less than the pivot’s to the left of the pivot and the other elements on its right, all quick_step() needs to do is call itself recursively on the two sections of the array. But note that quick_step() makes the recursive calls only for array sections that contain at least two elements.
Time to look at partition() , shown in Listing 9-10, which does the brunt of the sorting work.
Yes. It’s not simple. Let’s go through the logic of the algorithm from the beginning.
In lines 4 to 11, you define most of the working variables you need: k_pivot is the position of the pivot in the array (the shift-right-one-bit is a fancy way to divide an integer by 2); pivot points to the actual element; j1 is the index you use to scan the array from the left (set to k1-1 because you increment it in line 14 before using it); j2 is the index you use to scan the array from the right (set to k2+1 because you decrement it in line 15 before using it); swaps is a flag to remember whether you find two elements to swap while you execute an array sweep; and tmp is a variable you use to swap array elements.
Everything happens within the big do loop that starts in line 12 and ends in line 80. You keep it going as long as you find pairs of elements that you can swap. That is, one element before the pivot with a key greater than the pivot’s and another element after the pivot with a key less than the pivot’s. To check for this condition, you scan the array from the left until j1 hits an element with a key not less than the pivot’s (in line 14) and then you scan the array from the right until j2 hits an element with a key not greater than the pivot’s (line 15).
In other words, when you are past the loops in lines 14 and 15, you know that the key of ar[j1] is either equal or greater than the pivot’s, and the key of ar[j2] is either equal or less than the pivot’s. This generates four possible cases, which you handle as follows:
Case A is when both if statements in lines 16 and 17 succeed. Accordingly, in lines 18 to 20 you swap the two elements. You also set the flag swaps to true in line 21, so that the big do loop keeps looking for further swaps.
In any of the three remaining cases, you know that there cannot be any more swaps, because either you only have elements to be moved from the left of the pivot to its right (if i1 > 0) or from its right to its left (if i2 < 0), but not both. That is, you have to do the rest of the work in the current iteration of the big do loop without ever setting the variable swaps.
You handle case B in lines 24 to 48. Look at lines 24 and 26 to 34.
In line 24, you make a copy of the pivot’s index and name it kp. As do loops are always executed at least once, ignore for the moment that you are entering a loop in line 25.
You know from the check in line 16 that i1 > 0 and, assuming that the pivot does not coincide with the first element of the array (which would mean that kp == k1), you enter the while loop that starts in line 26.
In line 27, you decrement kp, so that it points to the element immediately on the left of the pivot. Then, in line 28, you compare that element with the pivot. You can safely reuse the variable i1 because it has completed its job to take you to the code associated with the appropriate case (in this instance, case B).
In line 29, you check whether the element in position kp (i.e., immediately on the left of the pivot) has a key greater than the pivot’s (i.e., whether i1 > 0). If that is the case, in lines 30 to 32 you swap the pivot with that element, thereby moving the pivot to the left.
The while loop in lines 26 to 34 keeps going until either you move the pivot so far left that it reaches the first position of the array (when kp == k1) or encounters an element with the key less than the pivot’s (when i1 < 0). Note that i1 cannot be 0 because all keys in the array are unique.
In any case, when you reach line 35, j1 still points to the element you needed to move to the right of the pivot but couldn’t because a swap was not possible, and kp points to the location on the left of the pivot, which contains an element with a key less than the pivot’s (kp doesn’t coincide with k_pivot because in the last iteration of the while loop, just before you exit it, the assignment made in line 32 is not executed).
In lines 36 to 38, you make a three-way swap: set the array element of the pivot to the element in j1 that needs to be moved to the right; set the j1 element to point to the element on the left of the pivot (which has a key less than the pivot’s); and, finally, write the pivot’s address one position to the left of where it was before the swap. And, obviously, in line 39, you update k_pivot.
So, after the three-way swap, kp and k_pivot point to the same location and j1 points to an element that is correctly located on the left of the pivot. But who’s to say that among the elements between j1 and kp (both excluded), there are no additional elements that should be moved to the right of the pivot?
To handle the situation, you repeat in line 43 the statement of line 14. That is, you move j1 to the right as long as it points to nodes that have keys less than the pivot’s. If, when you exit the tight loop in line 41, i1 is positive, it means that there is at least one element that should be moved to the right. As a result, the do condition in line 48 is satisfied and execution returns to line 26. Note that you don’t need to go back to the beginning of the big do loop that starts in line 12 because you already know that there are no elements on the right of the pivot that could be used for a swap.
Now, it can happen that when you reach line 35, all the elements between j1 and the position the pivot was in when you executed line 26 had keys greater than the pivot’s. In that case, kp equals j1 and you have already moved all those elements to the right of the pivot, including the element in j1. That is, you would have completed your partitioning. But you are not likely to exit the do loop of lines 25 to 48 because the while loop in lines 26 to 34 has left i1 set to either 0 or -1 and, at the same time, it is likely that kp > k1. Therefore, you can expect that the do condition in line 48 will still be satisfied and the loop will go through an additional iteration when you are actually done. This would have catastrophic consequences (read: a crash). The solution is to set i1 to a negative value if, when you reach line 35, kp <= j1.
Lines 53 to 77 handle case C. That is, they do for the elements on the right of the pivot what lines 24 to 48 do for the elements on the left.
To help you follow how the algorithm works its way through a real example, I sprinkled several printf() in the quicksort functions and executed the code shown in Listing 9-11.
fill_random() is the local function shown in Listing 9-12 and Sort_list() is a macro defined in sort.h and shown in Listing 9-13. The output, shown in Listing 9-14, includes all types of operations—swap, move pivot to the left, move pivot to the right, and three-way swap.
Notice how the string fmt is used to set the number of characters reserved for each printed number to the maximum number of digits that the number can have. That means the result is available in tabular form without using tabs and without wasting space when working with small arrays.
In order to use floor() and log10(), you need to include the standard header file math.h, but remember that the gcc linker doesn’t automatically include the C mathematical library. With Linux, you will need to add to the linker m as option -l and /usr/lib/x86_64-linux-gnu/ as option -L. In Eclipse, you can do it by opening the project’s properties and accessing the C/C++ Build > Settings > GCC C Linker > Libraries page.
Running Quicksort—The Output
The source file sort.c associated with this chapter includes all printf()s use to produce Listing 9-14. But keep in mind that it logs the element keys by typecasting the elements to (int *), which works only if the array consists of pointers to int (as in these examples) or if, at the least, the key components of the element structures are ints placed at the very beginning of the structures.
It is now time to check how efficient quicksort is when compared to the best Shell-sort we came up with earlier in this chapter. To do so, you can execute the code shown in Listing 9-15.
It turns out that quicksort requires on average 795 comparisons for a 100-element array, which is a much better result than the Shell-sort, which required 1313 comparisons. It might be argued that this good result is achieved at the expense of using a more complex algorithm, with more ifs and loops. But comparisons are expensive in terms of execution time because they involve the overhead of a callback function. Therefore, it makes some sense to count them as an indication of the algorithm’s efficiency.
Incidentally, you don’t need to pick as a pivot the middle element of an array. Indeed, traditionally, quicksort implementations choose as pivot the first or last elements. But choosing the middle element is likely to be a better choice. You can further improve the efficiency of the algorithm by choosing the median element of first, last, and middle array elements. To do so, you only need to insert after line 5 of partition() (Listing 9-10) the code shown in Listing 9-16.
As you execute it after the existing lines 4 and 5 of partition(), where you set the pivot to the middle element, you only need to change the pivot in four of the possible six ordering of the three values.
Incidentally, this addition was recommended by Sedgewick, whom you encountered earlier in this chapter. Anyhow, if you make this change, you increase the average number of comparisons for 100 elements from 795 to 831. But this is because the array is small, as the overhead of three additional comparisons for each invocation of partition() becomes less and less relevant when the number of elements in the array increases. Already with 1000 elements, the algorithm requires 12,461 comparisons without the added code and only 12,199 with the additional code.
Quicksort is so much better than the other algorithms, and having to sort arrays of numeric values is so common a task that I couldn’t resist making a version of the three functions sort_quick(), quick_step(), and partition() for arrays of integers instead of arrays of pointers.
I could have left it for you as an exercise, but I wanted to do it anyway. Anyhow, after doing it, I asked myself: shouldn’t it be possible to collapse the two cases B (i.e., move the pivot left) and C (i.e., move the pivot right) into a single piece of code? I found a way, and Listing 9-17 shows the resulting partition_int() function.
It is much more compact than the original version shown in Listing 9-10. The two big blocks of code in lines 24 to 48 and 53 to 77 have become line 24 and line 29, respectively. Notice that the two new lines are macros, so that you don’t introduce the overhead of calling functions. The Move_pivot_int() macro is shown in Listing 9-18.
As you can see, k_dir determines the direction in which you move the pivot, from left to right or from right to left, and also the “direction” of the if conditions. For example, when you are moving the pivot to the left in line 24 of the function, with jj set to j1 and k_dir set to -1, the condition in line 13 of the macro becomes:
or, removing the signs and inverting the direction of the comparison,
which is identical to the condition in line 35 of Listing 9-10 (and it well should be!).
Now, I don’t know about you, but I actually prefer partition_int() (and, obviously, the original partition() shown in Listing 9-10) without the macro. It does contain duplicated code, but I find it easier to understand. Indeed, it would have been too arduous a task to write directly the partition function with the macro. I already expended quite a bit of effort without the macro not to get confused with all the indices.
In the source code for this chapter, you will find two versions of the partition function for integer: one without the macro named partition_int() and one with the macro named partition_int_macro().
To complete the handling of arrays of integers, you find sort_quick_int(), quick_step_int(), and Sort_list_int() in Listings 9-19, 9-20, and 9-21, respectively. If you compare them to the corresponding versions for arrays of pointers (Listings 9-8, 9-9, and 9-13), you will see that the differences are minimal and predictable.
If you really love the Move_pivot_int() macro, I leave it up to you to make a Move_pivot() for partition().
The Standard C Function
The standard C library (stdlib) includes a function that applies quicksort to an array of pointers:
qsort() has two additional parameters: the length of the array (count) and the size of the array elements (size). They are not needed in sort_quick() because you store the array length before the data in the dynamically allocated array, and all the arrays consist of pointers to void.
The comparison function you use with qsort() returns 1 if the first argument is greater than the second, while the function used with sort_quick() does the opposite and returns 1 when the second argument is greater than the first. I should have used the same ordering of parameters to avoid any possible confusion. But it isn’t really an important difference.
I still wanted to show you how quicksort works, secretly hoping that my function would perform better than the function in the standard library. This is the time to compare the two. Listing 9-22 shows the code I used in order to do it. I used sort_quick_int() (Listing 9-19) and switched off all logging by setting both SORT_DEBUG and SORT_LOG to 0 in sort.h. I could have compared qsort() with sort_quick(), which handles arrays of pointers, instead of with sort_quick_int(), which doesn’t use a callback comparison function. But qsort() deals with arrays of objects like sort_quick_int(), rather than arrays of pointers to objects like sort_quick(), and only needs a callback function to be able to handle all sorts of numbers, rather than just integers.
Comparing qsort() and sort_quick_int()
The function cmp_i_qsort() is almost identical to cmp_i(), as shown in Listing 9-5. The only difference is that the order of the parameters is reversed.
Comparing qsort() and sort_quick_int()
With arrays of length 100, sort_quick_int() is 12.5% slower than qsort() but, as the number of elements increases, the percentage goes down to approximately 1.2%.
For fun, I increased the array size N to 100,000 and reduced N_REP to 2000 (to save time). It turns out that, for such big arrays, sort_quick_int() is 1.9% better than qsort(). We can definitely be happy with that!
In this chapter, you learned about all the common ways to sort arrays. You also saw how to measure their performance.