README 4.61 KB
Newer Older
matt russell's avatar
matt russell committed
1
2
3
4
5
6
7
unit_test
matt russell
comp15 2020 summer
updated 12/11/2020

unit_test - a frictionless C++ unit testing framework.

Matthew P. Russell's avatar
Matthew P. Russell committed
8
9
10
11
12
Note that the instructions are for use on the Tufts CS server, however unit_test
can be installed on your local system without issue. Simply use your own 
Python3.7 directory and edit ~/.cshrc or ~/.bashrc as indicated in the
instructions. 

Matthew P. Russell's avatar
Matthew P. Russell committed
13
-----------------------------------------------------------------------------
Matt Russell's avatar
Matt Russell committed
14
15
                Instructions for initial TA/Faculty set up
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
16
17
18
19
20

Somewhere on the homework server, run:

vm-hw01{utln}1001: git clone git@gitlab.cs.tufts.edu:mrussell/unit_test
vm-hw01{utln}1002: cd unit_test
Matt Russell's avatar
Matt Russell committed
21
22
vm-hw01{utln}1003: chmod u+x setup
vm-hw01{utln}1004: ./setup
matt russell's avatar
matt russell committed
23
24
25

You will be asked for the following:

Matt Russell's avatar
Matt Russell committed
26
    1) Python3.7 location       (Note: default is /g/15/2020uc/support/bin/python3.7)* 
Matt Russell's avatar
Matt Russell committed
27
28
    2) location for scripts     (Note: default is /comp/15/bin)**
    3) location for lib file(s) (Note: default is /comp/15/lib)**
matt russell's avatar
matt russell committed
29
30

*A working Python3.7 executable does live in that directory on the cs server; it
Matthew P. Russell's avatar
Matthew P. Russell committed
31
32
 will be available for 'all time', so please feel free to use the default! If
 you're on a local system, clearly you'll need a different path.
33

Matthew P. Russell's avatar
Matthew P. Russell committed
34
35
**If you want to set this up for personal use, rather than for the whole course
use something other than /comp/15/bin and /comp/15/lib.
matt russell's avatar
matt russell committed
36

Matthew P. Russell's avatar
Matthew P. Russell committed
37
38
39
Great! The unit testing framework is now set up. To actually run unit_test,
running the install script is still needed. The setup scrupt will ask you if
you'd like to do this. If you decline by accident, simply follow the steps below
matt russell's avatar
matt russell committed
40
------------------------------------------------------------------------------
Matt Russell's avatar
Matt Russell committed
41
42
                  Instructions for student installation
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
43

Matt Russell's avatar
Matt Russell committed
44
For students to install unit_test, they must run the install script
matt russell's avatar
matt russell committed
45

Matt Russell's avatar
Matt Russell committed
46
    /comp/15/bin/install_unit_test    
matt russell's avatar
matt russell committed
47

Matthew P. Russell's avatar
Matthew P. Russell committed
48
(replace /comp/15/bin with the /bin path specified during initial setup)
matt russell's avatar
matt russell committed
49

Matt Russell's avatar
Matt Russell committed
50
NOTE: do NOT use the install_unit_test located in the bin of this directory. 
Matthew P. Russell's avatar
Matthew P. Russell committed
51
NOTE: you CAN use the install_unit_test located in the 'build' directory.
Matt Russell's avatar
Matt Russell committed
52

Matthew P. Russell's avatar
Matthew P. Russell committed
53
54
After running the script, they should restart their shell,
or run: source ~/.cshrc
matt russell's avatar
matt russell committed
55

Matt Russell's avatar
Matt Russell committed
56
57
58

------------------------------------------------------------------------------
                    Instructions to run unit tests
59
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
60
61

To see the system in action:
Matt Russell's avatar
Matt Russell committed
62
   go to examples/
Matt Russell's avatar
Matt Russell committed
63
64
   run the command: unit_test (note: no './')

Matthew P. Russell's avatar
Matthew P. Russell committed
65
66
If installation has worked, tests should run. If things fail immediately,
confirm that all parhs were entered correctly, and that source ~/.cshrc was run
Matt Russell's avatar
Matt Russell committed
67

Matt Russell's avatar
Matt Russell committed
68
Note that the provided example has bugs, so the tests will not succeed. 
matt russell's avatar
matt russell committed
69
70
71
72
73

For other projects, adapt the Makefile and example_tests.cpp files.

See those files for details on how to update them.

Matt Russell's avatar
Matt Russell committed
74
75
76
77
78
79
80
Making the example files available to students is probably a good idea, but is
not done automatically during initialization


------------------------------------------------------------------------------
                      Notes for the brave of heart
------------------------------------------------------------------------------
matt russell's avatar
matt russell committed
81
82
83
84
85
86
87
88
89
90
91
92

There are a two key phases of this script: 

1) Initilize Autograding Files
2) Compile and Run (and Run Valgrind on) all test cases

The script also keeps track of student score, and prints results during runtime

Initialization: 

    This program works by leveraging an originally incomplete driver located at:

Matt Russell's avatar
Matt Russell committed
93
    lib/unit_test_driver.cpp
matt russell's avatar
matt russell committed
94
95
96

    It first copies this incomplete driver to the student's working directory.

97
98
99
    Then, it parses all files in the student's working directory with the suffix:

    	     '_tests.cpp'
matt russell's avatar
matt russell committed
100
101
102
103
104
105
106
107
108
109
110

    Specifically, for each such file, the script discovers all of the functions that have:
        1) a void return type
        2) no input arguments

    For each function, the script:
        1) Adds a forward declaration to the copied driver file. 
        2) Inserts a string in the driver file of the following format:
          
            { "test_function_name": test_function_name },\n

111
112
    This pair is inserted in the initialization of a std::map<std::string, FnPtr>> object,
    where "FnPtr" is a typdef of a void function that takes no arguments.
matt russell's avatar
matt russell committed
113
114
115

Compilation and Running:

Matt Russell's avatar
Matt Russell committed
116
117
118
119
    This code is relatively straightforward. After compilation, Python makes subprocess calls 
    to the compiled binary, passing in a testname as an argument.

    Valgrind is run automatically on all tests, and pass/fail results are displayed. 
matt russell's avatar
matt russell committed
120
121

    Passes / failures are kept track of; all results are printed to the terminal.