Package csb :: Package test
[frames] | no frames]

Source Code for Package csb.test

  1  """ 
  2  This is a top level package, hosting the entire CSB test framework. It is divided 
  3  into several major parts: 
  4   
  5      - test cases, located under csb.test.cases 
  6      - test data, in C{/csb/test/data} (not a package) 
  7      - test console, in C{/csb/test/app.py} 
  8   
  9  This module, csb.test, contains all the glue-code functions, classes and  
 10  decorators you would need in order to write tests for CSB.     
 11   
 12      1. Configuration and Tree 
 13       
 14         L{Config<csb.test.Config>} is a common config object shared between CSB 
 15         tests. Each config instance contains properties like: 
 16               
 17              - data: the data folder, automatically discovered and loaded in 
 18                csb.test.Config.DATA at module import time 
 19              - temp: a default temp folder, which test cases can use 
 20           
 21         Each L{Config<csb.test.Config>} provides a convenient way to retrieve 
 22         files from C{/csb/test/data}. Be sure to check out L{Config.getTestFile} 
 23         and L{Config.getPickle}. In case you need a temp file, use 
 24         L{Config.getTempStream} or have a look at L{csb.io.TempFile} and 
 25         L{csb.io.TempFolder}.  
 26           
 27         All test data files should be placed in the C{data} folder. All test 
 28         modules must be placed in the root package: csb.test.cases. There is 
 29         a strict naming convention for test modules: the name of a test module 
 30         should be the same as the name of the CSB API package it tests. For  
 31         example, if you are writing tests for C{csb/bio/io/__init__.py}, the 
 32         test module must be C{csb/test/cases/bio/io/__init__.py}. C{csb.test.cases} 
 33         is the root package of all test modules in CSB. 
 34       
 35      2. Writing Tests 
 36       
 37         Writing a test is easy. All you need is to import csb.test and then 
 38         create your own test cases, derived from L{csb.test.Case}: 
 39          
 40             >>> import csb.test 
 41             >>> @csb.test.unit 
 42                 class TestSomeClass(csb.test.Case): 
 43                     def setUp(self): 
 44                         super(TestSomeClass, self).setUp() 
 45                         # do something with self.config here... 
 46          
 47         In this way your test case instance is automatically equipped with a  
 48         reference to the test config, so your test method can be: 
 49   
 50             >>> @csb.test.unit 
 51                 class TestSomeClass(csb.test.Case): 
 52                     def testSomeMethod(self): 
 53                         myDataFile = self.config.getTestFile('some.file') 
 54                         self.assert... 
 55           
 56         The "unit" decorator marks a test case as a collection of unit tests. 
 57         All possibilities are: L{csb.test.unit}, L{csb.test.functional}, L{csb.test.custom}, 
 58         and L{csb.test.regression}. 
 59                      
 60         Writing custom (a.k.a. "data", "slow", "dynamic") tests is a little bit 
 61         more work. Custom tests must be functions, not classes. Basically a 
 62         custom test is a function, which builds a unittest.TestSuite instance  
 63         and then returns it when called without arguments. 
 64          
 65         Regression tests are usually created in response to reported bugs. Therefore,  
 66         the best practice is to mark each test method with its relevant bug ID: 
 67          
 68             >>> @csb.test.regression 
 69                 class SomeClassRegressions(csb.test.Case) 
 70                     def testSomeFeature(self) 
 71                     \""" 
 72                     @see: [CSB 000XXXX]  
 73                     \""" 
 74                     # regression test body... 
 75              
 76      3. Style Guide: 
 77       
 78         - name test case packages as already described 
 79         - group tests in csb.test.Case-s and name them properly 
 80         - prefix test methods with "test", like "testParser" - very important 
 81         - use camelCase for methods and variables. This applies to all the 
 82           code under csb.test (including test) and does not apply to the rest 
 83           of the library! 
 84         - for functional tests it's okay to define just one test method: runTest 
 85         - for unit tests you should create more specific test names, for example:  
 86           "testParseFile" - a unit test for some method called "parse_file" 
 87         - use csb.test decorators to mark tests as unit, functional, regression, etc. 
 88         - make every test module executable:: 
 89          
 90             if __name__ == '__main__': 
 91                 csb.test.Console()   # Discovers and runs all test cases in the module 
 92       
 93      4. Test Execution 
 94       
 95         Test discovery is handled by C{test builders} and a test runner 
 96         C{app}. Test builders are subclasses of L{AbstractTestBuilder}.   
 97         For every test type (unit, functional, regression, custom) there is a 
 98         corresponding test builder. L{AnyTestBuilder} is a special builder which 
 99         scans for unit, regression and functional tests at the same time. 
100   
101         Test builder classes inherit the following test discovery methods: 
102       
103             - C{loadTests} - load tests from a test namespace. Wildcard 
104               namespaces are handled by C{loadAllTests} 
105             - C{loadAllTests} - load tests from the given namespace, and 
106               from all sub-packages (recursive) 
107             - C{loadFromFile} - load tests from an absolute file name 
108             - C{loadMultipleTests} - calls C{loadTests} for a list of  
109               namespaces and combines all loaded tests in a single suite 
110                
111         Each of those return test suite objects, which can be directly executed 
112         with python's unittest runner. 
113          
114         Much simpler way to execute a test suite is to use our test app  
115         (C{csb/test/app.py}), which is simply an instance of L{csb.test.Console}:: 
116          
117             $ python csb/test/app.py --help 
118          
119         The app has two main arguments:  
120       
121             - test type - tells the app which TestBuilder to use for test dicsovery 
122               ("any" triggers L{AnyTestBuilder}, "unit" - L{UnitTestBuilder}, etc.)  
123             - test namespaces - a list of "dotted" test modules, for example:: 
124       
125                  csb.test.cases.bio.io.*   # io and sub-packages 
126                  csb.test.cases.bio.utils  # only utils 
127                  .                         # current module 
128       
129         In addition to running the app from the command line, you can run it 
130         also programmatically by instantiating L{csb.test.Console}. You can 
131         construct a test console object by passing a list of test namespace(s) 
132         and a test builder class to the Console's constructor. 
133   
134       
135      5. Commit Policies 
136       
137         Follow these guidelines when making changes to the repository: 
138       
139             - B{no bugs in "trunk"}: after fixing a bug or implementing a new 
140               feature, make sure at least the default test set passes by running 
141               the test console without any arguments. This is equivalent to: 
142               app.py -t any "csb.test.cases.*". (If no test case from this set covers 
143               the affected code, create a test case first, as described in the other 
144               policies) 
145       
146             - B{no recurrent issues}: when a bug is found, first write a regression 
147               test with a proper "@see: BugID" tag in the docstring. Run the test 
148               to make sure it fails. After fixing the bug, run the test again before 
149               you commit, as required by the previous policy 
150                
151             - B{test all new features}: there should be a test case for every new feature 
152               we implement. One possible approach is to write a test case first and 
153               make sure it fails; when the new feature is ready, run the test again 
154               to make sure it passes 
155   
156  @warning: for compatibility reasons do NOT import and use the unittest module 
157            directly. Always import unittest from csb.test, which is guaranteed 
158            to be python 2.7+ compatible. The standard unittest under python 2.6 
159            is missing some features, that's why csb.test will take care of 
160            replacing it with unittest2 instead.  
161  """ 
162  from __future__ import print_function 
163   
164  import os 
165  import sys 
166  import imp 
167  import types 
168  import time 
169  import getopt 
170  import tempfile 
171  import traceback 
172   
173  import csb.io 
174  import csb.core 
175   
176  try: 
177      from unittest import skip, skipIf 
178      import unittest 
179  except ImportError: 
180      import unittest2 as unittest 
181   
182  from abc import ABCMeta, abstractproperty 
183 184 185 -class Attributes(object):
186 187 UNIT = '__CSBUnitTest__' 188 CUSTOM = '__CSBCustomTest__' 189 FUNCTIONAL = '__CSBFunctionalTest__' 190 REGRESSION = '__CSBRegressionTest__'
191
192 -class Config(object):
193 """ 194 General CSB Test Config. Config instances contain the following properties: 195 196 - data - path to the CSB Test Data directory. Default is L{Config.DATA} 197 - temp - path to the system's temp directory. Default is L{Config.TEMP} 198 - config - the L{Config} class 199 """ 200 201 DATA = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data') 202 """ 203 @cvar: path to the default test data directory: <install dir>/csb/test/data 204 """ 205 TEMP = os.path.abspath(tempfile.gettempdir()) 206 """ 207 @cvar: path to the default system's temp directory 208 """ 209
210 - def __init__(self):
211 212 self.__config = Config 213 self.__data = Config.DATA 214 self.__temp = Config.TEMP
215 216 @staticmethod
217 - def setDefaultDataRoot(path):
218 """ 219 Override the default L{Config.DATA} with a new data root directory. 220 221 @param path: full directory path 222 @type path: str 223 """ 224 if not os.path.isdir(path): 225 raise IOError('Path not found: {0}'.format(path)) 226 227 Config.DATA = os.path.abspath(path)
228 229 @property
230 - def data(self):
231 """ 232 Test data directory 233 @rtype: str 234 """ 235 return self.__data
236 237 @property
238 - def temp(self):
239 """ 240 Test temp directory 241 @rtype: str 242 """ 243 return self.__temp
244
245 - def getTestFile(self, fileName, subDir=''):
246 """ 247 Search for C{fileName} in the L{Config.DATA} directory. 248 249 @param fileName: the name of a test file to retrieve 250 @type fileName: str 251 @param subDir: scan a sub-directory of L{Config.DATA} 252 @type subDir: str 253 254 @return: full path to C{fileName} 255 @rtype: str 256 257 @raise IOError: if no such file is found 258 """ 259 file = os.path.join(self.data, subDir, fileName) 260 if not os.path.isfile(file): 261 raise IOError('Test file not found: {0}'.format(file)) 262 return file
263
264 - def getPickle(self, fileName, subDir=''):
265 """ 266 Same as C{self.getTestFile}, but try to unpickle the data in the file. 267 268 @param fileName: the name of a test file to retrieve 269 @type fileName: str 270 @param subDir: scan a sub-directory of L{Config.DATA} 271 @type subDir: str 272 """ 273 file = self.getTestFile(fileName, subDir) 274 return csb.io.Pickle.load(open(file, 'rb'))
275
276 - def getContent(self, fileName, subDir=''):
277 """ 278 Same as C{self.getTestFile}, but also read and return the contents of 279 the file. 280 281 @param fileName: the name of a test file to retrieve 282 @type fileName: str 283 @param subDir: scan a sub-directory of L{Config.DATA} 284 @type subDir: str 285 """ 286 with open(self.getTestFile(fileName, subDir)) as f: 287 return f.read()
288
289 - def getTempStream(self, mode='t'):
290 """ 291 Return a temporary file stream:: 292 293 with self.getTempStream() as tmp: 294 tmp.write(something) 295 tmp.flush() 296 file_name = tmp.name 297 298 @param mode: file open mode (text, binary), default=t 299 @type mode: str 300 @rtype: file stream 301 """ 302 return csb.io.TempFile(mode=mode)
303
304 - def ensureDataConsistency(self):
305 """ 306 Try to deserialize some pickled data files. Call L{Config.updateDataFiles} 307 if the pickles appeared incompatible with the current interpreter. 308 """ 309 try: 310 self.getPickle('1nz9.model1.pickle') 311 except: 312 self.updateDataFiles()
313
314 - def updateDataFiles(self):
315 """ 316 Refresh the pickled structures in csb/test/data. This might be needed when 317 the internal representation of some classes has changed. 318 """ 319 from csb.io import Pickle 320 from csb.bio.io.wwpdb import RegularStructureParser 321 from csb.bio.structure import Ensemble, ChemElements 322 323 parser = RegularStructureParser(self.getTestFile('1nz9.pdb')) 324 model1 = parser.parse_structure(model=1) 325 model2 = parser.parse_structure(model=2) 326 327 ensemble = Ensemble() 328 ensemble.models.append(model1) 329 ensemble.models.append(model2) 330 Pickle.dump(ensemble, open(os.path.join(self.data, '1nz9.full.pickle'), 'wb')) 331 332 mse = model1.chains['A'].find(164) 333 mse._pdb_name = 'MSE' 334 mse.atoms['SD']._element = ChemElements.Se 335 mse.atoms['SD']._full_name = 'SE ' 336 Pickle.dump(model1, open(os.path.join(self.data, '1nz9.model1.pickle'), 'wb'))
337
338 -class Case(unittest.TestCase):
339 """ 340 Base class, defining a CSB Test Case. Provides a default implementation 341 of C{unittest.TestCase.setUp} which grabs a reference to a L{Config}. 342 """ 343 344 @property
345 - def config(self):
346 """ 347 Test config instance 348 @rtype: L{Config} 349 """ 350 return self.__config
351
352 - def setUp(self):
353 """ 354 Provide a reference to the CSB Test Config in the C{self.config} property. 355 """ 356 self.__config = Config() 357 assert hasattr(self.config, 'data'), 'The CSB Test Config must contain the data directory' 358 assert self.config.data, 'The CSB Test Config must contain the data directory'
359
360 - def reRaise(self, addArgs=()):
361 """ 362 Re-raise the last exception with its full traceback, but modify the 363 argument list with C{addArgs} and the original stack trace. 364 365 @param addArgs: additional arguments to append to the exception 366 @type addArgs: tuple 367 """ 368 klass, ex, _tb = sys.exc_info() 369 ex.args = list(ex.args) + list(addArgs) + [''.join(traceback.format_exc())] 370 371 raise klass(ex.args)
372
373 - def assertAlmostEqual(self, first, second, places=None, msg=None, delta=None):
374 375 if first == second: 376 return 377 if delta is not None and places is not None: 378 raise TypeError("specify delta or places not both") 379 380 if delta is not None: 381 382 if abs(first - second) <= delta: 383 return 384 385 m = '{0} != {1} within {2} delta'.format(first, second, delta) 386 msg = self._formatMessage(msg, m) 387 388 raise self.failureException(msg) 389 390 else: 391 if places is None: 392 places = 7 393 394 return super(Case, self).assertAlmostEqual(first, second, places=places, msg=msg)
395
396 - def assertFasterThan(self, duration, callable, *args, **kargs):
397 """ 398 Fail if it took more than C{duration} seconds to invoke C{callable}. 399 400 @param duration: maximum amount of seconds allowed 401 @type duration: float 402 """ 403 404 start = time.time() 405 callable(*args, **kargs) 406 execution = time.time() - start 407 408 if execution > duration: 409 self.fail('{0}s is slower than {1}s)'.format(execution, duration))
410 411 @classmethod
412 - def execute(cls):
413 """ 414 Run this test case. 415 """ 416 suite = unittest.TestLoader().loadTestsFromTestCase(cls) 417 runner = unittest.TextTestRunner() 418 419 return runner.run(suite)
420
421 -class InvalidNamespaceError(NameError, ImportError):
422 pass
423
424 -class AbstractTestBuilder(object):
425 """ 426 This is a base class, defining a test loader which exposes the C{loadTests} 427 method. 428 429 Subclasses must override the C{labels} abstract property, which controls 430 what kind of test cases are loaded by the test builder. 431 """ 432 433 __metaclass__ = ABCMeta 434 435 @abstractproperty
436 - def labels(self):
437 pass
438
439 - def loadFromFile(self, file):
440 """ 441 Load L{csb.test.Case}s from a module file. 442 443 @param file: test module file name 444 @type file: str 445 446 @return: a C{unittest.TestSuite} ready for the test runner 447 @rtype: C{unittest.TestSuite} 448 """ 449 mod = self._loadSource(file) 450 suite = unittest.TestLoader().loadTestsFromModule(mod) 451 return unittest.TestSuite(self._filter(suite))
452
453 - def loadTests(self, namespace):
454 """ 455 Load L{csb.test.Case}s from the given CSB C{namespace}. If the namespace 456 ends with a wildcard, tests from sub-packages will be loaded as well. 457 If the namespace is '__main__' or '.', tests are loaded from __main__. 458 459 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will 460 load tests from '/csb/test/cases/bio/__init__.py' 461 @type namespace: str 462 463 @return: a C{unittest.TestSuite} ready for the test runner 464 @rtype: C{unittest.TestSuite} 465 """ 466 if namespace.strip() == '.*': 467 namespace = '__main__.*' 468 elif namespace.strip() == '.': 469 namespace = '__main__' 470 471 if namespace.endswith('.*'): 472 return self.loadAllTests(namespace[:-2]) 473 else: 474 loader = unittest.TestLoader() 475 tests = loader.loadTestsFromName(namespace) 476 return unittest.TestSuite(self._filter(tests))
477
478 - def loadMultipleTests(self, namespaces):
479 """ 480 Load L{csb.test.Case}s from a list of given CSB C{namespaces}. 481 482 @param namespaces: a list of test module namespaces, e.g. 483 ('csb.test.cases.bio', 'csb.test.cases.bio.io') will 484 load tests from '/csb/test/cases/bio.py' and 485 '/csb/test/cases/bio/io.py' 486 @type namespaces: tuple of str 487 488 @return: a C{unittest.TestSuite} ready for the test runner 489 @rtype: C{unittest.TestSuite} 490 """ 491 if not csb.core.iterable(namespaces): 492 raise TypeError(namespaces) 493 494 return unittest.TestSuite(self.loadTests(n) for n in namespaces)
495
496 - def loadAllTests(self, namespace, extension='.py'):
497 """ 498 Load L{csb.test.Case}s recursively from the given CSB C{namespace} and 499 all of its sub-packages. Same as:: 500 501 builder.loadTests('namespace.*') 502 503 @param namespace: test module namespace, e.g. 'csb.test.cases.bio' will 504 load tests from /csb/test/cases/bio/*' 505 @type namespace: str 506 507 @return: a C{unittest.TestSuite} ready for the test runner 508 @rtype: C{unittest.TestSuite} 509 """ 510 suites = [] 511 512 try: 513 base = __import__(namespace, level=0, fromlist=['']).__file__ 514 except ImportError: 515 raise InvalidNamespaceError('Namespapce {0} is not importable'.format(namespace)) 516 517 if os.path.splitext(os.path.basename(base))[0] != '__init__': 518 suites.append(self.loadTests(namespace)) 519 520 else: 521 522 for entry in os.walk(os.path.dirname(base)): 523 524 for item in entry[2]: 525 file = os.path.join(entry[0], item) 526 if extension and item.endswith(extension): 527 suites.append(self.loadFromFile(file)) 528 529 return unittest.TestSuite(suites)
530
531 - def _loadSource(self, path):
532 """ 533 Import and return the Python module identified by C{path}. 534 535 @note: Module objects behave as singletons. If you import two different 536 modules and give them the same name in imp.load_source(mn), this 537 counts for a redefinition of the module originally named mn, which 538 is basically the same as reload(mn). Therefore, you need to ensure 539 that for every call to imp.load_source(mn, src.py) the mn parameter 540 is a string that uniquely identifies the source file src.py. 541 """ 542 name = os.path.splitext(os.path.abspath(path))[0] 543 name = name.replace('.', '-').rstrip('__init__').strip(os.path.sep) 544 545 return imp.load_source(name, path)
546
547 - def _recurse(self, obj):
548 """ 549 Extract test cases recursively from a test C{obj} container. 550 """ 551 cases = [] 552 if isinstance(obj, unittest.TestSuite) or csb.core.iterable(obj): 553 for item in obj: 554 cases.extend(self._recurse(item)) 555 else: 556 cases.append(obj) 557 return cases
558
559 - def _filter(self, tests):
560 """ 561 Filter a list of objects using C{self.labels}. 562 """ 563 filtered = [] 564 565 for test in self._recurse(tests): 566 for label in self.labels: 567 if hasattr(test, label) and getattr(test, label) is True: 568 filtered.append(test) 569 570 return filtered
571
572 -class AnyTestBuilder(AbstractTestBuilder):
573 """ 574 Build a test suite of cases, marked as either unit, functional or regression 575 tests. For detailed documentation see L{AbstractTestBuilder}. 576 """ 577 @property
578 - def labels(self):
580
581 -class UnitTestBuilder(AbstractTestBuilder):
582 """ 583 Build a test suite of cases, marked as unit tests. 584 For detailed documentation see L{AbstractTestBuilder}. 585 """ 586 @property
587 - def labels(self):
588 return [Attributes.UNIT]
589
590 -class FunctionalTestBuilder(AbstractTestBuilder):
591 """ 592 Build a test suite of cases, marked as functional tests. 593 For detailed documentation see L{AbstractTestBuilder}. 594 """ 595 @property
596 - def labels(self):
597 return [Attributes.FUNCTIONAL]
598
599 -class RegressionTestBuilder(AbstractTestBuilder):
600 """ 601 Build a test suite of cases, marked as regression tests. 602 For detailed documentation see L{AbstractTestBuilder}. 603 """ 604 @property
605 - def labels(self):
606 return [Attributes.REGRESSION]
607
608 -class CustomTestBuilder(AbstractTestBuilder):
609 """ 610 Build a test suite of cases, marked as custom tests. CustomTestBuilder will 611 search for functions, marked with the 'custom' test decorator, which return 612 a dynamically built C{unittest.TestSuite} object when called without 613 parameters. This is convenient when doing data-related tests, e.g. 614 instantiating a single type of a test case many times iteratively, for 615 each entry in a database. 616 617 For detailed documentation see L{AbstractTestBuilder}. 618 """ 619 @property
620 - def labels(self):
621 return [Attributes.CUSTOM]
622
623 - def loadFromFile(self, file):
624 625 mod = self._loadSource(file) 626 suites = self._inspect(mod) 627 628 return unittest.TestSuite(suites)
629
630 - def loadTests(self, namespace):
631 632 if namespace.strip() == '.*': 633 namespace = '__main__.*' 634 elif namespace.strip() == '.': 635 namespace = '__main__' 636 637 if namespace.endswith('.*'): 638 return self.loadAllTests(namespace[:-2]) 639 else: 640 try: 641 mod = __import__(namespace, fromlist=['']) 642 except ImportError: 643 raise InvalidNamespaceError('Namespace {0} is not importable'.format(namespace)) 644 suites = self._inspect(mod) 645 return unittest.TestSuite(suites)
646
647 - def _inspect(self, module):
648 649 objects = map(lambda n: getattr(module, n), dir(module)) 650 return self._filter(objects)
651
652 - def _filter(self, factories):
653 """ 654 Filter a list of objects using C{self.labels}. 655 """ 656 filtered = [] 657 658 for obj in factories: 659 for label in self.labels: 660 if hasattr(obj, label) and getattr(obj, label) is True: 661 suite = obj() 662 if not isinstance(suite, unittest.TestSuite): 663 raise ValueError('Custom test function {0} must return a ' 664 'unittest.TestSuite, not {1}'.format(obj.__name__, type(suite))) 665 filtered.append(suite) 666 667 return filtered
668
669 -def unit(klass):
670 """ 671 A class decorator, used to label unit test cases. 672 673 @param klass: a C{unittest.TestCase} class type 674 @type klass: type 675 """ 676 if not isinstance(klass, type): 677 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 678 679 setattr(klass, Attributes.UNIT, True) 680 return klass
681
682 -def functional(klass):
683 """ 684 A class decorator, used to label functional test cases. 685 686 @param klass: a C{unittest.TestCase} class type 687 @type klass: type 688 """ 689 if not isinstance(klass, type): 690 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 691 692 setattr(klass, Attributes.FUNCTIONAL, True) 693 return klass
694
695 -def regression(klass):
696 """ 697 A class decorator, used to label regression test cases. 698 699 @param klass: a C{unittest.TestCase} class type 700 @type klass: type 701 """ 702 if not isinstance(klass, type): 703 raise TypeError("Can't apply class decorator on {0}".format(type(klass))) 704 705 setattr(klass, Attributes.REGRESSION, True) 706 return klass
707
708 -def custom(function):
709 """ 710 A function decorator, used to mark functions which build custom (dynamic) 711 test suites when called. 712 713 @param function: a callable object, which returns a dynamically compiled 714 C{unittest.TestSuite} 715 @type function: callable 716 """ 717 if isinstance(function, type): 718 raise TypeError("Can't apply function decorator on a class") 719 elif not hasattr(function, '__call__'): 720 raise TypeError("Can't apply function decorator on non-callable {0}".format(type(function))) 721 722 setattr(function, Attributes.CUSTOM, True) 723 return function
724
725 -def skip(reason, condition=None):
726 """ 727 Mark a test case or method for skipping. 728 729 @param reason: message 730 @type reason: str 731 @param condition: skip only if the specified condition is True 732 @type condition: bool/expression 733 """ 734 if isinstance(reason, types.FunctionType): 735 raise TypeError('skip: no reason specified') 736 737 if condition is None: 738 return unittest.skip(reason) 739 else: 740 return unittest.skipIf(condition, reason)
741
742 -class Console(object):
743 """ 744 Build and run all tests of the specified namespace and kind. 745 746 @param namespace: a dotted name, which specifies the test module 747 (see L{csb.test.AbstractTestBuilder.loadTests}) 748 @type namespace: str 749 @param builder: test builder to use 750 @type builder: any L{csb.test.AbstractTestBuilder} subclass 751 @param verbosity: verbosity level for C{unittest.TestRunner} 752 @type verbosity: int 753 @param update: if True, refresh all pickles in csb/test/data 754 @type update: bool 755 """ 756 757 BUILDERS = {'unit': UnitTestBuilder, 'functional': FunctionalTestBuilder, 758 'custom': CustomTestBuilder, 'any': AnyTestBuilder, 759 'regression': RegressionTestBuilder} 760 761 USAGE = r""" 762 CSB Test Runner Console. Usage: 763 764 python {0.program} [-u] [-t type] [-v verbosity] namespace(s) 765 766 Options: 767 namespace(s) A list of CSB test dotted namespaces, from which to 768 load tests. '__main__' and '.' are interpreted as the 769 current module. If a namespace ends with an asterisk 770 '.*', all sub-packages will be scanned as well. 771 772 Examples: 773 "csb.test.cases.bio.*" 774 "csb.test.cases.bio.io" "csb.test.cases.bio.utils" 775 "." 776 777 -t type Type of tests to load from each namespace. Possible 778 values are: 779 {0.builders} 780 781 -v verbosity Verbosity level passed to unittest.TextTestRunner. 782 783 -u update-files Force update of the test pickles in csb/test/data. 784 """ 785
786 - def __init__(self, namespace=('__main__',), builder=AnyTestBuilder, verbosity=1, 787 update=False, argv=None):
788 789 if not argv: 790 argv = sys.argv 791 792 self._namespace = None 793 self._builder = None 794 self._verbosity = 1 795 self._update = False 796 self._program = os.path.basename(argv[0]) 797 798 self.namespace = namespace 799 self.builder = builder 800 self.verbosity = verbosity 801 self.update = update 802 803 self.parseArguments(argv[1:]) 804 self.run()
805 806 @property
807 - def namespace(self):
808 return self._namespace
809 @namespace.setter
810 - def namespace(self, value):
811 if csb.core.iterable(value): 812 self._namespace = list(value) 813 else: 814 self._namespace = [value]
815 816 @property
817 - def builder(self):
818 return self._builder
819 @builder.setter
820 - def builder(self, value):
821 self._builder = value
822 823 @property
824 - def verbosity(self):
825 return self._verbosity
826 @verbosity.setter
827 - def verbosity(self, value):
828 self._verbosity = value
829 830 @property
831 - def builders(self):
832 return ', '.join(Console.BUILDERS)
833 834 @property
835 - def program(self):
836 return self._program
837 838 @property
839 - def update(self):
840 return self._update
841 @update.setter
842 - def update(self, value):
843 self._update = bool(value)
844
845 - def run(self):
846 847 if self.update: 848 Config().updateDataFiles() 849 else: 850 Config().ensureDataConsistency() 851 852 builder = self.builder() 853 suite = builder.loadMultipleTests(self.namespace) 854 855 runner = unittest.TextTestRunner(verbosity=self.verbosity) 856 runner.run(suite)
857
858 - def exit(self, message=None, code=0, usage=True):
859 860 if message: 861 print(message) 862 if usage: 863 print(Console.USAGE.format(self)) 864 865 sys.exit(code)
866
867 - def parseArguments(self, argv):
868 869 try: 870 871 options, args = getopt.getopt(argv, 'hut:v:', ['help', 'update-files', 'type=', 'verbosity=']) 872 873 for option, value in options: 874 if option in('-h', '--help'): 875 self.exit(message=None, code=0) 876 if option in('-t', '--type'): 877 try: 878 self.builder = Console.BUILDERS[value] 879 except KeyError: 880 self.exit(message='E: Invalid test type "{0}".'.format(value), code=2) 881 if option in('-v', '--verbosity'): 882 try: 883 self.verbosity = int(value) 884 except ValueError: 885 self.exit(message='E: Verbosity must be an integer.', code=3) 886 if option in('-u', '--update-files'): 887 self.update = True 888 889 if len(args) > 0: 890 self.namespace = list(args) 891 892 except getopt.GetoptError as oe: 893 self.exit(message='E: ' + str(oe), code=1)
894 895 896 if __name__ == '__main__': 897 898 Console() 899