README 4.61 KB
Newer Older
matt russell's avatar
matt russell committed
1 2 3 4 5 6 7
unit_test
matt russell
comp15 2020 summer
updated 12/11/2020

unit_test - a frictionless C++ unit testing framework.

Matthew P. Russell's avatar
Matthew P. Russell committed
8 9 10 11 12
Note that the instructions are for use on the Tufts CS server, however unit_test
can be installed on your local system without issue. Simply use your own 
Python3.7 directory and edit ~/.cshrc or ~/.bashrc as indicated in the
instructions. 

Matthew P. Russell's avatar
Matthew P. Russell committed
13
-----------------------------------------------------------------------------
Matt Russell's avatar
Matt Russell committed
14 15
                Instructions for initial TA/Faculty set up
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
16 17 18 19 20

Somewhere on the homework server, run:

vm-hw01{utln}1001: git clone git@gitlab.cs.tufts.edu:mrussell/unit_test
vm-hw01{utln}1002: cd unit_test
Matt Russell's avatar
Matt Russell committed
21 22
vm-hw01{utln}1003: chmod u+x setup
vm-hw01{utln}1004: ./setup
matt russell's avatar
matt russell committed
23 24 25

You will be asked for the following:

Matt Russell's avatar
Matt Russell committed
26
    1) Python3.7 location       (Note: default is /g/15/2020uc/support/bin/python3.7)* 
Matt Russell's avatar
Matt Russell committed
27 28
    2) location for scripts     (Note: default is /comp/15/bin)**
    3) location for lib file(s) (Note: default is /comp/15/lib)**
matt russell's avatar
matt russell committed
29 30

*A working Python3.7 executable does live in that directory on the cs server; it
Matthew P. Russell's avatar
Matthew P. Russell committed
31 32
 will be available for 'all time', so please feel free to use the default! If
 you're on a local system, clearly you'll need a different path.
33

Matthew P. Russell's avatar
Matthew P. Russell committed
34 35
**If you want to set this up for personal use, rather than for the whole course
use something other than /comp/15/bin and /comp/15/lib.
matt russell's avatar
matt russell committed
36

Matthew P. Russell's avatar
Matthew P. Russell committed
37 38 39
Great! The unit testing framework is now set up. To actually run unit_test,
running the install script is still needed. The setup scrupt will ask you if
you'd like to do this. If you decline by accident, simply follow the steps below
matt russell's avatar
matt russell committed
40
------------------------------------------------------------------------------
Matt Russell's avatar
Matt Russell committed
41 42
                  Instructions for student installation
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
43

Matt Russell's avatar
Matt Russell committed
44
For students to install unit_test, they must run the install script
matt russell's avatar
matt russell committed
45

Matt Russell's avatar
Matt Russell committed
46
    /comp/15/bin/install_unit_test    
matt russell's avatar
matt russell committed
47

Matthew P. Russell's avatar
Matthew P. Russell committed
48
(replace /comp/15/bin with the /bin path specified during initial setup)
matt russell's avatar
matt russell committed
49

Matt Russell's avatar
Matt Russell committed
50
NOTE: do NOT use the install_unit_test located in the bin of this directory. 
Matthew P. Russell's avatar
Matthew P. Russell committed
51
NOTE: you CAN use the install_unit_test located in the 'build' directory.
Matt Russell's avatar
Matt Russell committed
52

Matthew P. Russell's avatar
Matthew P. Russell committed
53 54
After running the script, they should restart their shell,
or run: source ~/.cshrc
matt russell's avatar
matt russell committed
55

Matt Russell's avatar
Matt Russell committed
56 57 58

------------------------------------------------------------------------------
                    Instructions to run unit tests
59
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
60 61

To see the system in action:
Matt Russell's avatar
Matt Russell committed
62
   go to examples/
Matt Russell's avatar
Matt Russell committed
63 64
   run the command: unit_test (note: no './')

Matthew P. Russell's avatar
Matthew P. Russell committed
65 66
If installation has worked, tests should run. If things fail immediately,
confirm that all parhs were entered correctly, and that source ~/.cshrc was run
Matt Russell's avatar
Matt Russell committed
67

Matt Russell's avatar
Matt Russell committed
68
Note that the provided example has bugs, so the tests will not succeed. 
matt russell's avatar
matt russell committed
69 70 71 72 73

For other projects, adapt the Makefile and example_tests.cpp files.

See those files for details on how to update them.

Matt Russell's avatar
Matt Russell committed
74 75 76 77 78 79 80
Making the example files available to students is probably a good idea, but is
not done automatically during initialization


------------------------------------------------------------------------------
                      Notes for the brave of heart
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
81 82 83 84 85 86 87 88 89 90 91 92

There are a two key phases of this script: 

1) Initilize Autograding Files
2) Compile and Run (and Run Valgrind on) all test cases

The script also keeps track of student score, and prints results during runtime

Initialization: 

    This program works by leveraging an originally incomplete driver located at:

Matt Russell's avatar
Matt Russell committed
93
    lib/unit_test_driver.cpp
matt russell's avatar
matt russell committed
94 95 96

    It first copies this incomplete driver to the student's working directory.

97 98 99
    Then, it parses all files in the student's working directory with the suffix:

    	     '_tests.cpp'
matt russell's avatar
matt russell committed
100 101 102 103 104 105 106 107 108 109 110

    Specifically, for each such file, the script discovers all of the functions that have:
        1) a void return type
        2) no input arguments

    For each function, the script:
        1) Adds a forward declaration to the copied driver file. 
        2) Inserts a string in the driver file of the following format:
          
            { "test_function_name": test_function_name },\n

111 112
    This pair is inserted in the initialization of a std::map<std::string, FnPtr>> object,
    where "FnPtr" is a typdef of a void function that takes no arguments.
matt russell's avatar
matt russell committed
113 114 115

Compilation and Running:

Matt Russell's avatar
Matt Russell committed
116 117 118 119
    This code is relatively straightforward. After compilation, Python makes subprocess calls 
    to the compiled binary, passing in a testname as an argument.

    Valgrind is run automatically on all tests, and pass/fail results are displayed. 
matt russell's avatar
matt russell committed
120 121

    Passes / failures are kept track of; all results are printed to the terminal.