More tests

[jmid]

This PR

• adds tuple and bind tests (both positive, negative, and statistics).
• I also added a manual shrinker and test for the IntTree test.
• I grouped test-names per sub-module as the long list at the end was getting hard to maintain.

Edit: I've now also

• added shrink-logging tests to compare the total number of shrinking attempts.
• separated the tests into a separate "test library" and runner.
  Then we have one runner for the expect tests in the CI - while other runners can reuse (some of) the same tests,
  e.g., for local shrinker benchmarking.

Here's the diff -y of the new test outputs. Note how QCheck2's int shrinking strategy generally spends
less successful shrinking steps (again) - as have been discussed in, e.g., PR #153 and #173:

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs have different components failed (0 shrink steps):		Test pairs have different components failed (0 shrink steps):

(4, 4)									(4, 4)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs have same components failed (125 shrink steps):	     |	Test pairs have same components failed (63 shrink steps):

(0, 1)									(0, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs have a zero component failed (124 shrink steps):	     |	Test pairs have a zero component failed (122 shrink steps):

(-1, 1)								     |	(1, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs are (0,0) failed (125 shrink steps):			     |	Test pairs are (0,0) failed (63 shrink steps):

(0, 1)									(0, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs are ordered failed (125 shrink steps):		     |	Test pairs are ordered failed (2 shrink steps):

(0, -1)									(0, -1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs are ordered reversely failed (125 shrink steps):	     |	Test pairs are ordered reversely failed (63 shrink steps):

(0, 1)									(0, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test pairs sum to less than 128 failed (121 shrink steps):	     |	Test pairs sum to less than 128 failed (59 shrink steps):

(0, 128)								(0, 128)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test triples have pair-wise different components failed (7 shrink st |	Test triples have pair-wise different components failed (3 shrink st

(0, 7, 7)							     |	(0, 0, 0)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test triples have same components failed (188 shrink steps):	     |	Test triples have same components failed (64 shrink steps):

(0, -1, 0)							     |	(0, 1, 0)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test triples are ordered failed (188 shrink steps):		     |	Test triples are ordered failed (3 shrink steps):

(0, -1, 0)								(0, -1, 0)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test triples are ordered reversely failed (188 shrink steps):	     |	Test triples are ordered reversely failed (64 shrink steps):

(0, 0, 1)								(0, 0, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test quadruples have pair-wise different components failed (23 shrin |	Test quadruples have pair-wise different components failed (4 shrink

(0, 0, 0, 0)								(0, 0, 0, 0)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test quadruples have same components failed (250 shrink steps):	     |	Test quadruples have same components failed (126 shrink steps):

(0, 1, 0, 1)								(0, 1, 0, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test quadruples are ordered failed (251 shrink steps):		     |	Test quadruples are ordered failed (5 shrink steps):

(0, 0, -1, 0)								(0, 0, -1, 0)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test quadruples are ordered reversely failed (251 shrink steps):     |	Test quadruples are ordered reversely failed (66 shrink steps):

(0, 0, 0, 1)								(0, 0, 0, 1)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test bind ordered pairs failed (123 shrink steps):		     |	Test bind ordered pairs failed (1 shrink steps):

(0, 0)									(0, 0)

--- Failure --------------------------------------------------------	--- Failure --------------------------------------------------------

Test bind list_size constant failed (261 shrink steps):		     |	Test bind list_size constant failed (15 shrink steps):

(4, [0; 0; 0; 0])							(4, [0; 0; 0; 0])

As a bonus, there are also pretty (and completely identical) histograms to be found:

+++ Stats for quad dist ++++++++++++++++++++++++++++++++++++++++++++	+++ Stats for quad dist ++++++++++++++++++++++++++++++++++++++++++++

stats quad sum:								stats quad sum:
  num: 500000, avg: 200.13, stddev: 58.33, median 200, min 5, max 39	  num: 500000, avg: 200.13, stddev: 58.33, median 200, min 5, max 39
    5.. 24:                                                         	    5.. 24:                                                         
   25.. 44:                                                         	   25.. 44:                                                         
   45.. 64: ##                                                      	   45.. 64: ##                                                      
   65.. 84: ######                                                  	   65.. 84: ######                                                  
   85..104: ############                                            	   85..104: ############                                            
  105..124: #####################                                   	  105..124: #####################                                   
  125..144: ###############################                         	  125..144: ###############################                         
  145..164: ##########################################              	  145..164: ##########################################              
  165..184: ##################################################      	  165..184: ##################################################      
  185..204: ####################################################### 	  185..204: ####################################################### 
  205..224: #####################################################   	  205..224: #####################################################   
  225..244: ###############################################         	  225..244: ###############################################         
  245..264: ######################################                  	  245..264: ######################################                  
  265..284: ##########################                              	  265..284: ##########################                              
  285..304: ################                                        	  285..304: ################                                        
  305..324: #########                                               	  305..324: #########                                               
  325..344: ####                                                    	  325..344: ####                                                    
  345..364: #                                                       	  345..364: #                                                       
  365..384:                                                         	  365..384:                                                         
  385..404:                                                         	  385..404:                                                         

+++ Stats for bind dist ++++++++++++++++++++++++++++++++++++++++++++	+++ Stats for bind dist ++++++++++++++++++++++++++++++++++++++++++++

stats ordered pair difference:						stats ordered pair difference:
  num: 1000000, avg: 25.02, stddev: 22.36, median 19, min 0, max 100	  num: 1000000, avg: 25.02, stddev: 22.36, median 19, min 0, max 100
    0..  4: ####################################################### 	    0..  4: ####################################################### 
    5..  9: #####################################                   	    5..  9: #####################################                   
   10.. 14: #############################                           	   10.. 14: #############################                           
   15.. 19: ########################                                	   15.. 19: ########################                                
   20.. 24: #####################                                   	   20.. 24: #####################                                   
   25.. 29: ##################                                      	   25.. 29: ##################                                      
   30.. 34: ################                                        	   30.. 34: ################                                        
   35.. 39: #############                                           	   35.. 39: #############                                           
   40.. 44: ############                                            	   40.. 44: ############                                            
   45.. 49: ##########                                              	   45.. 49: ##########                                              
   50.. 54: #########                                               	   50.. 54: #########                                               
   55.. 59: ########                                                	   55.. 59: ########                                                
   60.. 64: ######                                                  	   60.. 64: ######                                                  
   65.. 69: #####                                                   	   65.. 69: #####                                                   
   70.. 74: ####                                                    	   70.. 74: ####                                                    
   75.. 79: ###                                                     	   75.. 79: ###                                                     
   80.. 84: ##                                                      	   80.. 84: ##                                                      
   85.. 89: ##                                                      	   85.. 89: ##                                                      
   90.. 94: #                                                       	   90.. 94: #                                                       
   95.. 99:                                                         	   95.. 99:                                                         
  100..104:                                                         	  100..104:                                                         

stats ordered pair sum:							stats ordered pair sum:
  num: 1000000, avg: 75.12, stddev: 46.93, median 72, min 0, max 200	  num: 1000000, avg: 75.12, stddev: 46.93, median 72, min 0, max 200
    0..  9: ####################################################### 	    0..  9: ####################################################### 
   10.. 19: #####################################################   	   10.. 19: #####################################################   
   20.. 29: #####################################################   	   20.. 29: #####################################################   
   30.. 39: #####################################################   	   30.. 39: #####################################################   
   40.. 49: #####################################################   	   40.. 49: #####################################################   
   50.. 59: #####################################################   	   50.. 59: #####################################################   
   60.. 69: #####################################################   	   60.. 69: #####################################################   
   70.. 79: #####################################################   	   70.. 79: #####################################################   
   80.. 89: #####################################################   	   80.. 89: #####################################################   
   90.. 99: #####################################################   	   90.. 99: #####################################################   
  100..109: ##################################################      	  100..109: ##################################################      
  110..119: ###########################################             	  110..119: ###########################################             
  120..129: #####################################                   	  120..129: #####################################                   
  130..139: ###############################                         	  130..139: ###############################                         
  140..149: #########################                               	  140..149: #########################                               
  150..159: ####################                                    	  150..159: ####################                                    
  160..169: ###############                                         	  160..169: ###############                                         
  170..179: ###########                                             	  170..179: ###########                                             
  180..189: ######                                                  	  180..189: ######                                                  
  190..199: ##                                                      	  190..199: ##                                                      
  200..209:                                                         	  200..209:                                                         

Edit: This again builds on top of #172 and #174 (merge! merge! 😄)

[jmid]

A few observations:

1. I found myself wanting a uniform positive int generator. With QCheck I can just write (pair pos_int pos_int), e.g., in the test pair_ordered but for QCheck2 to achieve the same I have to write Gen.(pair (pint ~origin:0) (pint ~origin:0)). The opaque Gen.t makes the optional parameter mandatory - which is just clunky (also pointed out in issue QCheck2.Gen design considerations #162)

2. QCheck2's int-shrinker relies on bind, as it first generates a bool to decide the integer's sign. This has a side-effect for shrinking: true (meaning "generate a negative int") is reduced to false, thus reducing negative integers to positive ones - which can seem simpler as an end-user. Since we don't use a splittable RNG, the Random.State has moved on since the original state, and therefore the resulting int shrinking strategy will reduce an arbitrary negative integer -1975781842156211842 to an arbitrary positive integer 2696939011544317271 (not necessarily with a smaller, absolute value):

   $ head shrink_algo_logs/triple_same_components_qcheck2.expected 
   fails (-1975781842156211842, -4571327332697646483, -3285013039971199785) 
   fails (2696939011544317271, -4571327332697646483, -3285013039971199785)
   fails (0, -4571327332697646483, -3285013039971199785)
   fails (0, 3095744455229849699, -3285013039971199785)
   holds (0, 0, -3285013039971199785)
   ...
   

   This is also how 4571327332697646483 is reduced to the seemingly unrelated 3095744455229849699 above.

3. The lack of a splittable RNG also makes for an unpredictable strategy when list and pair generators are combined in QCheck2. Thus in shrink_algo_logs/pair_lists_rev_concat_qcheck2.expected we find, e.g.:

   ...
   fails ([3762171117042495591; 4588024816217148396], [2696939011544317271; 1975781842156211841; 1035416029544138122; 1118378519987614091])
   holds ([], [2696939011544317271; 1975781842156211841; 1035416029544138122; 1118378519987614091])
   fails ([2152069955941623198], [2696939011544317271; 1975781842156211841; 1035416029544138122; 1118378519987614091])
   holds ([], [2696939011544317271; 1975781842156211841; 1035416029544138122; 1118378519987614091])
   fails ([0], [2696939011544317271; 1975781842156211841; 1035416029544138122; 1118378519987614091])
   holds ([0], [])
   fails ([0], [1599388225294475516; 3378876932193098527])
   ...
   

   Here a 2-element list in the first component is reduced to a seemingly unrelated 1-element list, and later a 4-element list in the second component is reduced to a seemingly unrelated 2-element list.
   This strategy of "starting from Random.State scratch" affects the result when the generator has been lucky to find a Random.State producing a counterexample requiring some relation between the components of a tuple, e.g., for
   Test pairs lists no overlap QCheck returns ([0], [0]) after 22 successful shrink steps whereas QCheck2 returns ([0], [0; 0; 0; 0]) after 27 successful shrink steps.

Overall:

• the design discussion in QCheck2.Gen design considerations #162 is still relevant (@sir4ur0n, @vch9, @c-cube, ...)
• we should consider adopting the "negative-to-positive reduction strategy" in our QCheck int-shrinkers
• we should consider adopting the "reduce-to-0-first strategy" in our QCheck int-shrinkers as the shrinking steps of irrelevant ints add up in composite generators (this is an ongoing discussion with @Gbury in QCheck: improve int shrinker fast path #173)
• we should add a splittable RNG (Add a splittable random number generator #86) for more predictable QCheck2 shrinking. As I've mentioned previously I have a poor-man's version here: https://github.com/jmid/sm2-tes21/blob/5c1d9406b8457560999cf031c0c808011590bf70/lec10/intqc.ml#L66-L69 which I plan to try out when these observable tests are in place, to better assess the impact.

[jmid]

This PR is superseded by #234 and #237