Formal engineering of the bitonic sort using PVS

In this paper, we present a proof that the bitonic sort is sound using PVS, a powerful specification and verification environment. First, we briefly introduce this well-known parallel sort. It is based on bitonic lists whose relevant properties can be proven with PVS. To achieve our goal of constructing the proof from scratch, we start by studying some examples of this sort. Then we try to prove properties of this algorithm. Failure in the proof of particular lemmas provides us with information which helps to correct these lemmas. To complete this proof, we start with general cases, continue by examining each of the exception cases, and finish when all cases have been considered. Then we can construct the specification of the bitonic sort which can easily be translated into a traditional imperative language.


Introduction
Proof of (parallel) programs can be done using different techniques and formalisms.We mention for example UNITY [4] and TLA [11], chosen from a very long list of possibilities [15].Some techniques are based on states, others on behaviors or on actions, but none of them allows us to build and prove 'correct' all the different kinds of possible programs (parallel or not).Some of them use high level concepts and are well-suited for high level specifications, but in general they make unrealistic assumptions (such as unlimited process resource, or unbounded communication rates,...).
In this paper, we try only to identify some properties concerned with steps of induction on the natural recursion of the algorithm.Usually, proofs of this algorithm assume the property that subsequences of a bitonic sequence are themselves bitonic.We have not casually adopted such an assumption.In our proof, we require a similar property but we prove all such assumptions before using them.Similarly, for other problems, we advocate proceeding in the same fully rigorous manner.
To obtain a property, we first try to elaborate an idea of it.Then we formalize the idea using PVS.At this stage, some ideas were suppressed because they were not expressible.Also, some expressible properties were not provable because they required undesirable hypothesis: in these cases we must reformulate a property until it can be proven.Finally, when we have all the necessary properties, we must verify that all different non-common cases are handled.To do this we found PVS to be a powerful tool because it detects all such cases, whereas humans are prone to missing some of them.
In section two, we state the problem and give an example which illustrates the bitonic sort.Section three describes the PVS system.In section four, we start by proving simple properties useful for the proof.Then we prove all the properties required to show that this algorithm is correct.In fact, these properties are based on an induction step of this recursive algorithm.Section five presents the specification of the bitonic sort using the formalism of PVS.In section six we briefly present other work in this domain; and finally in section seven we conclude.

Problem
In this section, we explain the problem, and we present some well-accepted definitions.The bitonic sort, a fast parallel sort [1], uses a bitonic list which can be defined as follows: 2nd Irish Workshop on Formal Methods, 1998 that each half of the sequence can be sorted in parallel since comparison between elements is independent.
The first line is the sequence unsorted.At first, we consider all pairs of numbers as bitonic lists with an increasing sequence of one element followed by a decreasing sequence of one element.With each pair, we merge both the bitonic lists (the increasing and the decreasing one) in order to obtain alternatively an increasing list and a decreasing list, each of 2 numbers.The second line of the figure shows the sequence after the first merge.With the pair (13,3), we wish to have an increasing list and with the pair (95, 5) we wish to have a decreasing list.
The next step (lines 3 and 4) consists of making bitonic sequences of 4 elements, having alternatively increasing sequences and decreasing sequences of 2 elements.This is, in fact, achieved in 2 substeps.For each sequence, we apply the algorithm called bitonic split (there are two versions depending on whether we wish to have an increasing or a decreasing list).
2nd Irish Workshop on Formal Methods, 1998 Consider, for instance, the bitonic list (3, 13, 95, 5) (these numbers are the four first numbers of the second line).Here we wish to have an increasing sequence with the same elements.To do this, we compare element 1 and element 3 (¿ ½ • ¾ ¾ ½ ) and put respectively the minimum and maximum at indexes 1 and 3.So 3 and 95 keep the same places.Applying the same method at indexes 2 and 4 (with numbers 13 and 5), there is a permutation, so we obtain the sequences (3,5) and (95, 13) (line 3).Using the recursivity of the algorithm, we obtain 2 sequences (3, 5) and (13, 95) (line 4).So the initial sequence is, as we wished, sorted.
At line 4, we have alternatively increasing and decreasing sequences of 4 numbers.Taking the eight first numbers, we must use the bitonic merge algorithm in order to yield an increasing list of 8 numbers.The merge is made at line 5, using the same algorithm previously described.Applying the same techniques, we have at line 7, one increasing list (of 8 numbers) followed by one decreasing list; and finally at line 11, we have the sorted sequence containing all numbers.

How to prove the correctness of this algorithm
In this section, we make explicit some properties that we want to prove, although we have not yet described the specification of the bitonic sort.These properties follow directly from the previous example; but we will see they are not sufficient to verify this algorithm.
As we have seen in Definition 2.3, it seems to be interesting to see if a bitonic split applied to a bitonic list yields two bitonic lists.Furthermore, as we have seen in the example, we use the same algorithm for Definition 2.2.Thus we should be looking to see if this definition can be proven to be a theorem.Keep in mind that we have, as yet, no ideas on how to prove these properties and can only hope that they are sufficient to verify the bitonic sort algorithm.
Concerning increasing and decreasing lists, we took into account the symmetry inherent in the problem to factor out some complexity.It seemed easier to prove the required properties with increasing lists and use an algorithm to reverse an increasing sequence in order to have a decreasing sequence.
Formal engineering of the bitonic sort using PVS Other useful properties will be introduced in the proof, in Section 4.

PVS
PVS (Prototype Verification System) is a powerful specification and verification environment [6,13].It provides tools to create and analyse formal specifications, and to prove theorems interactively.It has been used in various domains such as avionics [7], verification of reactive systems [10,14] and program design [9].
The PVS specification language is based on typed higher-order logic; it has a rich type system including constructors, dependent types, abstract data types and a subtyping mechanism.These features allow us to express powerful specifications, and its type checker helps us to avoid many semantic errors.Large specifications can be split into different files to form a hierarchy of re-usable theories.
The PVS prover provides a set of inbuilt steps that can be used to simplify the current goal that can be discharged automatically by the prover.The system automatically builds proof obligations called TCCs (Type Checking Conditions) and it is able to prove most of them without any user intervention.Simple theorems are proved automatically with defined tactics of PVS, but with more difficult theorems, the user must drive the proof with help of the interactive prover.The PVS theorem prover is based on calculus sequent, so proving a theorem consists of proving that the equivalent sequent is true.During a proof, the current sequent -the goal -undergoes transformations which either completes it or generates subgoals which then have to be proven.PVS provides parameterizable commands which give the user various ways of completing a proof.
Each sequent transformation is made with a pre-defined command of PVS.The system contains more than a hundred commands.These commands are more or less complex.In general, we try to apply high level commands such as induction, automatic rewrite rules, instantiation with heuristics....With experience, we learn the capabilities of these commands.After application of a sequence of such commands, we complete the proof with powerful simplification and decision procedures for linear arithmetic and equality.All the commands can be combined to form proof strategies, allowing several proof rules to be applied in one step.

Proof of interesting properties with PVS
In this section, we explain the process by which we proved the bitonic sort.We note that all aspects of the proof were constructed from scratch.This should explain why this algorithm proof took approximately one man-month to complete.

Proof of basic properties
Here we present some type definitions for use during the proof.We need an array : The first line defines a new type Arra, an array ( we do not use the word Array because it is a key-word of PVS).The second and the third lines define some variables (we do not use long variable names because, during proofs we use these names several times and it is time consuming, when interacting with the prover, to write the same words repetitively).Thus x, y, i and j are variables that we use with FORALL and EXISTS as instantiated variables of universal quantifiers.nb is the size of the current array and nbp1 represents half of nb.We consider that, after splitting, an array of size nb gives 2 arrays of size nbp with nbp=nb/2.hi and low are respectively the inferior and the superior bounds of our array.i is a pivot in the array such that: before it, elements are increasing; and after it, elements are decreasing.j is another pivot which we will explain later.A and Ap are respectively the array at the current state and next states.
With these basic definitions we define : Formal engineering of the bitonic sort using PVS IncreasingList(A,low,hi) : bool = FORALL x,y:x>=low and y<=hi and x<=y => A(x)<=A(y) DecreasingList(A,low,hi) : bool = FORALL x,y:x>=low and y<=hi and x<=y => A(x)>=A(y) These definitions are formulas (or functions) for PVS.They return a boolean value.The first one, for instance, is interpreted as : For all x and y such that x is superior or equal to low, such that y is inferior or equal to hi and such that x is inferior or equal to y, this implies that the value of A at index x is inferior or equal to the value of A at index y.
Next we define the general case of a bitonic list (other cases will be considered later): BitonicList(A,low,hi,i) : bool = i>low and i<=hi and IncreasingList(A,low,i-1) and DecreasingList(A,i,hi) So we have an inceasing list from low to i-1 and a decreasing list from i to hi.So j is an index between low+1 and low+nbp-1, such that it splits a general bitonic list as follows.Elements between low and j are inferior to elements between low+nbp and j+nbp, and elements between j+nbp and low+nbp are superior to elements between j+nbp and hi (see figure 2).Without the two last lines (FORALL x ...), we could have undesirable situations such as is illustrated in figure 3. We represent an increasing (decreasing) list by an ascending (descending) segment but we do not suppose that numbers are uniformly increasing (decreasing).These schemas are just visual abstractions of a bitonic sequence: we don't give any scale.For example, sequences (4,5,12,30), (4,20,28,30) and (4,10,20,30) have the same visual abstraction.

Remark 4.2
In the previous version of this specification we omitted both the lines starting with (FORALL x ...) in the definition of Pivot.Trying to prove the following theorems with PVS, we could not complete these theorems because PVS did not make the natural human hypothesis that j is a number of the increasing list.Thus, without using such a theorem prover, our specification would have contained an implicit human hypothesis.Now that we have defined i and j, we can prove properties on places i and j.Here are the theorems we will use in the proof:  (i>=j or ( A(i)=A(i+nbp) and A(i)>=A(i-1) and j>i)) Figure 4 shows properties of theorems TRY 0, TRY 1, TRY 2 and TRY 3. The bold line shows the set of x and the neighbouring label is the property of the theorem.For example, figure 4(a) shows that for all Ü such as Ü Ð Ó Û and Ü ½, then Ü ½ (it is exactly as specified in lemma TRY 0).Theorem TRY 4 is an assembly of theorems TRY 0, TRY 1, TRY 2 and TRY 3. We constuct a theorem TRY 6, similar to theorem TRY 5, in which we put i<=j+nbp or (A(i-1)=A(i-nbp-1) and A(i)<=A(i-1) and i>j+nbp in place of i>=j or ( A(i)=A(i+nbp) and A(i)>=A(i-1) and j>i.Theorems TRY 5 and TRY 6 make explicit the different possible places of i and j (see figure 5).
Next we specify the minmax algorithm which plays the role of both merge and split (in 2.1 we saw that they were equivalent).

Proof that the obtained lists are bitonic ones
Most of the following proofs are done inductively on the recursive algorithm.Thus we need to prove only that one step is correct.In these cases, we suppose we have a list of size nb and the induction gives us two lists of size nbp=nb/2.The induction stops when nbp is equal to one.Now we can try to prove that the first list obtained with the minmax algorithm, applied on a general case of a bitonic list, is a bitonic list too.Using the theorem previously proved as an hypothesis, we want to conclude that there exists a new value j in the list after the minmax algorithm such that j is the pivot of the obtained bitonic list.Figures 6 and 7 show 2 different cases obtained after application of the minmax algorithm.
First we require another formula specifying all the different relative positions of i and j: Place_ij(A,i,j,nbp): bool = i>0 and j>0 and (i>=j or (A(i)=A(i+nbp) and A(i)>=A(i-1) and j>i)) and (i<=j+nbp or (A(i-1)=A(i-nbp-1) and A(i)<=A(i-1) and i>j+nbp )) and The proof of the theorem BIT 0 is long and tedious (though not difficult) since doing expansions of formulas and instantiations to solve subgoals requires time.
We must now prove that the second list obtained with the minmax algorithm is also a bitonic sequence.Regarding figures 6 and 7, the sequences formed by sets C and B do not seem to be bitonic sequences.In Definition 2.1 we saw that a bitonic sequence can be a left circular permutation of a general bitonic sequence (an increasing sequence followed by a decreasing sequence).
2nd Irish Workshop on Formal Methods, 1998   ,s) ) The variable s is the shift of a left circular permutation, nx and ny are new values of x and y, the quantified variables in FORALL.
The formulas IncreasingList2 and DecreasingList2 are the new definitions of an increasing and a decreasing list in which we allow left circular permutations of their elements.If x plus the shift s is superior or equal to half of the initial list (low+nbp), then the new value of x, nx is equal to x+s else nx is equal to x+s+nbp.In both these cases, nx has a value in the second half of the list since only the second list obtained can be a left circular permutation of a bitonic list.
New definitions of increasing and decreasing lists involve a new definition of a bitonic list (BitonicList2).At each (sub)step, the exact value of the shift is s=j-low.

Remark 4.3 In the previous version of the specification we omitted both the last lines in the definition of Biton-icList2.
Trying to prove the theorem BIT 1, we failed because we did not consider the cases where the second obtained list could be only increasing or decreasing.
This proof is by far the most difficult because there are several different cases.We list the different cases which we had to consider.The possible positions of i and j gives three cases (see figure 5).Since we use a new formula for a bitonic list (BitonicList2), able to handle the special case of the permutation of the second obtained list, we have three different cases, namely an increasing list followed by a decreasing list, only an increasing list or only a decreasing list.Both non-common cases (only increasing or decreasing) are respectively obtained when i=j+nbp and i=j.Both these cases are not very different from both non-common cases obtained with position of i and j (obtained with conditions: j>i and j+nbp<i) but we must nevertheless prove them.In the general case of a bitonic list (not only increasing or not only decreasing), we must prove that we have an increasing list followed by a decreasing list, so this gives two more cases.Having an increasing or a decreasing list, there are three different cases depending on the values of nx and ny.These cases are nx=x+s and ny=y+s, nx=x+s and ny=y+s+nbp, and nx=x+s+nbp and ny=y+s+nbp.In total, we see that the combination of all these cases gives 54 combinations.
Another big problem for this proof is due to the fact that each terminal subgoal (for which there exists no other dependant subgoals directly provable) contains an average of between thirty and forty formulas, three quarters being inequations and equations.In this situation PVS is not able to prove these subgoals with simple commands so we are constrained to telling PVS which formulas to use in order to complete the subgoals.This is quite boring but no more so than a proof "by hand".
2nd Irish Workshop on Formal Methods, 1998

Proof that the permutation of a bitonic list is equivalent to a bitonic one with the minmax algorithm
We have just seen that the second list obtained by the minmax algorithm is a left circular permutation of a bitonic one.Now we must prove that applying recursively the algorithm on a bitonic list and on a left circular permutation gives the same result.We prove that at any step of the recursion, if the shift is superior than the size of the obtained list then the next value of the shift is decreased by the size of the list.This method is equivalent to saying that the new value of the shift is equal to the precedent value modulo the size of the obtained list.It is obvious that by applying this method we obtain, at the end of the recursion, the same list in both cases.
To prove this we need to define new variables: sam,sp,spam : VAR nat SA,SAp : VAR Arra The variable sam is the value of the shift, since we add it to a variable and we need to apply a modulo on it, we call it "shift after modulo".sp and spam are respectively the values of s and sam in the next state.SA is an array containing the elements of A after the shifting and SAp is the next state of this array.
We first define some simple theorems (R is a relation on the naturals): The formula SHIFT PROP is the synthesis of the four previous theorems.The formula SHIFT is used by the formulas SHIFT , ¾ ¼ ¿ which use different comparison operators (>=, <=, > and <).For instance, the theorem SHIFT 0 means that if A(x)=SA(x+sam), i.e.SA is a left circular permutation of A, and if all elements in the first half of A are superior or equal to all elements of the second one, then we know the position of elements in SA, which depends on the value x+sam<low+nbp.So with these properties, we can build both the following theorems.The first one deals with the first half of the list whilst the second one deals with the second half.Both theorems are quite difficult to prove because of the several cases involved.Figure 8 shows that by shifting the second half of the list obtained after the minmax algorithm we obtain a general bitonic list.The meaning of the lemma SHIFT2 is: if the sequence SA is a left circular permutation of the sequence A (the shift being s) and if we apply the minmax algorithm on both the sequences A and SA, then the new shift (ns) on the second list obtained is equal to s modulo nbp and Ap is a left circular permuation of the sequence SAp.

SHIFT
Applying recursively the properties on sequences, the size of s decreases and after the last step its value is zero.It is obvious that the previous version of this lemma could not be proved.
In the derivation of the sequent of this lemma (the previous one) we obtain, after having hidden all the other formulas : Through analysis of this result we saw that we must replace x<=hi by x<=low+nbp.

End of the proof
As we have produced only increasing sequences with this specification, we prove that reversing an increasing list produces a decreasing list.This theorem is easy to prove: Another fundamental property to prove is that all elements in the first list, obtained after the minmax algorithm, are inferior to all the elements of the second one.This property, being applied recursively, yields a correct sort of the array, because at the end, the array is composed of lists of length one, and each element is obviously inferior to its successors.With this theorem we specify that if we have a bitonic list with suitable pivots i and j and if we apply the minmax algorithm on the sequence then all elements x in the first list obtained are inferior to all elements y of the second one.Now there are some special cases, not yet handled, which we should consider.
2nd Irish Workshop on Formal Methods, 1998 Thus, with both cases (A(j-1)>A(j-1+nbp) or A(j)<A(j+nbp)), we must prove properties similar to those seen in the normal case (where j>low and j<=low+nbp-1), i.e. we must prove that applying the minmax algorithm we obtain bitonic sequences in which all elements of the first obtained list are inferior to all elements of the second one.
The theorem PROP NOJ 0 is used to conclude that if we have a bitonic list and if all elements indexed by x in the first half of the sequence are superior to all elements indexed by x+nbp in the second half then for all x and y, indexes of the second half, such that x is inferior or equal to y then we have A(x) superior or equal to A(y).We use this property, in the following theorem, to prove that in the case where A(j-1)>A(j-1+nbp), we obtain a bitonic list (theorem BIT NOJ) in which we have the inferior property (theorem ¯Having a bitonic list ¾ Ô and applying our algorithm, we obtain two bitonic lists of size ¾ Ô ½ (a general bitonic list and a left circular permutation of a general bitonic list).
¯All the elements in the first bitonic list obtained are inferior to all the elements in the second one.¯A left circular permutation of a bitonic list is equivalent, at the end of a recursion of the minmax algorithm, to a bitonic one.
Thus starting with a list of size one and making, at each step, sequences of size two times bigger (alternatively increasing and decreasing), we obtain at the end of the algorithm an increasing list of size ¾ Ô .Thus, using all these theorems, we can ensure that the following specification is correct.

Specification of the bitonic sort algorithm in PVS
Now that we have proved all the properties that the specification should ensure, we can give the specification of the sort algorithm.To build this specification, we must examine how this algorithm executes.It starts by creating alternatively increasing and decreasing lists of size 2. Then at each step Ô, it builds alternatively increasing and decreasing lists of size ¾ Ô .Each step Ô being composed of Ô substeps.Thus we need to have a recursive function to merge, at each step Ô, an increasing and a decreasing list of size ¾ Ô ½ , in order to build either an increasing or decreasing list.But as we have seen, the algorithm requires Ô substeps to do this task.Thus the merge function needs to call a function which splits recursively a list of size ¾ Ô into two lists of size ¾ Ô ½ .These functions are respectively called BitonicMerge and BitonicSplit.
The function BitonicMerge, is called at the begining of the algorithm with n cur = 2.This value is the size of the first lists built by the algorithm.Then at each recursion, the value of n cur is multiplied by 2, until it reachs the value of nb.Alternatively we build an increasing list and a decreasing list using the function Reverse.
The function BitonicSplit simply splits a list into two lists using the BitonicMin and BitonicMax functions.Bilardi and Nicolau in [2] give some properties of bitonic sequences, but they use properties proven by other people; for example, they use the fact that a subsequence of a bitonic list is bitonic.
Concerning properties of programs, several work has already been done.In UNITY [3], Chandy and Misra defined a general framework in which stating programs, properties and mapping shows how formal techniques could be put together in a uniform notation.The concept of refinement is a very crucial point in the UNITY philosophy: informally, a text refines another text when 'what is holding' for the first text is 'still holding' for the second text.

Conclusion
We prove that the bitonic sort algorithm is sound using only basic knowledge of this algorithm.In fact we start by studying this well known algorithm, and describing (formally) its main properties.Then we proved these properties to be correct: PVS was a great help for this.Most of the properties used during the proof were not identified before the proof begin.In fact, we formulated draft versions of the properties, and it is only through trying to prove them that we were able to achieve correct definitions.Developing the proof of an algorithm in parallel with the specification of the algorithm is a novel, yet powerful, approched to software development.
This work is an illustration of program properties that can be proven with help of an interactive theorem prover.In [5], we explain a methodology we develop to parallelize a sequential application and prove that this parallelization is sound using post-conditions of the sequential program.We apply this method on a Monte Carlo simulation developed by our Physicist colleagues.We intend to test our approach on many more algorithms.

Figure 2 :
Figure 2: Examples of the bitonic sort

Figure 5 :
Figure 5: Different places of i and j

Figure 8 :
Figure 8: Shifting the second list, we obtain a general bitonic list

Remark 4 . 4
Now we give an example of a mistake in one of our previous versions of the specification.Since we are in the middle of elaborating the property SHIFT, the final part of the previous version of this lemma was FORALL x,spam:x>=low and x<=hi and spam = IF x+sp>low+nbp THEN sp-nbp ELSE sp ENDIF => Ap(x)=SAp(x+spam) in place of FORALL x,spam:x>=low and x<low+nbp and spam=IF x+sp>=low+nbp THEN sp-nbp ELSE sp ENDIF => Ap(x)=SAp(x+spam)