The –report-refcounts (-r) option can be used with the –repeat (-N) option to detect and diagnose memory leaks. To use this option, you must configure Python with the –with-pydebug option. (On Unix, pass this option to configure and then build Python.)
>>> import os.path, sys
>>> directory_with_tests = os.path.join(this_directory, 'testrunner-ex')
>>> defaults = [
... '--path', directory_with_tests,
... '--tests-pattern', '^sampletestsf?$',
... ]
>>> from zope import testrunner
>>> sys.argv = 'test --layer Layer11$ --layer Layer12$ -N4 -r'.split()
>>> _ = testrunner.run_internal(defaults)
Running samplelayers.Layer11 tests:
Set up samplelayers.Layer1 in 0.000 seconds.
Set up samplelayers.Layer11 in 0.000 seconds.
Iteration 1
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.013 seconds.
Iteration 2
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.012 seconds.
sys refcount=100401 change=0
Iteration 3
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.012 seconds.
sys refcount=100401 change=0
Iteration 4
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.013 seconds.
sys refcount=100401 change=0
Running samplelayers.Layer12 tests:
Tear down samplelayers.Layer11 in 0.000 seconds.
Set up samplelayers.Layer12 in 0.000 seconds.
Iteration 1
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.013 seconds.
Iteration 2
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.012 seconds.
sys refcount=100411 change=0
Iteration 3
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.012 seconds.
sys refcount=100411 change=0
Iteration 4
Ran 26 tests with 0 failures, 0 errors and 0 skipped in 0.012 seconds.
sys refcount=100411 change=0
Tearing down left over layers:
Tear down samplelayers.Layer12 in 0.000 seconds.
Tear down samplelayers.Layer1 in 0.000 seconds.
Total: 68 tests, 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Each layer is repeated the requested number of times. For each iteration after the first, the system refcount and change in system refcount is shown. The system refcount is the total of all refcount in the system. When a refcount on any object is changed, the system refcount is changed by the same amount. Tests that don’t leak show zero changes in systen refcount.
Let’s look at an example test that leaks:
>>> sys.argv = 'test --tests-pattern leak -N4 -r'.split()
>>> _ = testrunner.run_internal(defaults)
Running zope.testrunner.layer.UnitTests tests:...
Iteration 1
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
Iteration 2
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sys refcount=92506 change=12
Iteration 3
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sys refcount=92513 change=12
Iteration 4
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sys refcount=92520 change=12
Tearing down left over layers:
Tear down zope.testrunner.layer.UnitTests in N.NNN seconds.
Here we see that the system refcount is increating. If we specify a verbosity greater than one, we can get details broken out by object type (or class):
>>> sys.argv = 'test --tests-pattern leak -N5 -r -v'.split()
>>> _ = testrunner.run_internal(defaults)
Running tests at level 1
Running zope.testrunner.layer.UnitTests tests:...
Iteration 1
Running:
.
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
Iteration 2
Running:
.
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sum detail refcount=95832 sys refcount=105668 change=16
Leak details, changes in instances and refcounts by type/class:
type/class insts refs
------------------------------------------------------- ----- ----
classobj 0 1
dict 2 2
float 1 1
int 2 2
leak.ClassicLeakable 1 1
leak.Leakable 1 1
str 0 4
tuple 1 1
type 0 3
------------------------------------------------------- ----- ----
total 8 16
Iteration 3
Running:
.
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sum detail refcount=95844 sys refcount=105680 change=12
Leak details, changes in instances and refcounts by type/class:
type/class insts refs
------------------------------------------------------- ----- ----
classobj 0 1
dict 2 2
float 1 1
int -1 0
leak.ClassicLeakable 1 1
leak.Leakable 1 1
str 0 4
tuple 1 1
type 0 1
------------------------------------------------------- ----- ----
total 5 12
Iteration 4
Running:
.
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sum detail refcount=95856 sys refcount=105692 change=12
Leak details, changes in instances and refcounts by type/class:
type/class insts refs
------------------------------------------------------- ----- ----
classobj 0 1
dict 2 2
float 1 1
leak.ClassicLeakable 1 1
leak.Leakable 1 1
str 0 4
tuple 1 1
type 0 1
------------------------------------------------------- ----- ----
total 6 12
Iteration 5
Running:
.
Ran 1 tests with 0 failures, 0 errors and 0 skipped in 0.000 seconds.
sum detail refcount=95868 sys refcount=105704 change=12
Leak details, changes in instances and refcounts by type/class:
type/class insts refs
------------------------------------------------------- ----- ----
classobj 0 1
dict 2 2
float 1 1
leak.ClassicLeakable 1 1
leak.Leakable 1 1
str 0 4
tuple 1 1
type 0 1
------------------------------------------------------- ----- ----
total 6 12
Tearing down left over layers:
Tear down zope.testrunner.layer.UnitTests in N.NNN seconds.
It is instructive to analyze the results in some detail. The test being run was designed to intentionally leak:
- class ClassicLeakable:
- def __init__(self):
- self.x = ‘x’
- class Leakable(object):
- def __init__(self):
- self.x = ‘x’
leaked = []
class TestSomething(unittest.TestCase):
- def testleak(self):
- leaked.append((ClassicLeakable(), Leakable(), time.time()))
Let’s go through this by type.
The summary statistics include the sum of the detail refcounts. (Note that this sum is less than the system refcount. This is because the detailed analysis doesn’t inspect every object. Not all objects in the system are returned by sys.getobjects.)