I have a repository that contains multiple programs:

.
└── Programs
    ├── program1
    │   └── Generic_named.py
    └── program2
        └── Generic_named.py

I would like to add testing to this repository.

I have attempted to do it like this:

.
├── Programs
│   ├── program1
│   │   └── Generic_named.py
│   └── program2
│       └── Generic_named.py
└── Tests
    ├── mock
    │   ├── 1
    │   │   └── custom_module.py
    │   └── 2
    │       └── custom_module.py
    ├── temp
    ├── test1.py
    └── test2.py

Where temp is a folder to store each program temporarily with mock versions of any required imports that can not be stored directly with the program.

Suppose we use a hello world example like this:

cat Programs/program1/Generic_named.py
import custom_module

def main():
    return custom_module.out()


cat Programs/program2/Generic_named.py
import custom_module

def main():
    return custom_module.out("Goodbye, World!")


cat Tests/mock/1/custom_module.py
def out():return "Hello, World!"


cat Tests/mock/2/custom_module.py
def out(x):return x

And I were to use these scripts to test it:

cat Tests/test1.py
import unittest
import os
import sys
import shutil

if os.path.exists('Tests/temp/1'):
    shutil.rmtree('Tests/temp/1')

shutil.copytree('Tests/mock/1', 'Tests/temp/1/')
shutil.copyfile('Programs/program1/Generic_named.py', 'Tests/temp/1/Generic_named.py')

sys.path.append('Tests/temp/1')
import Generic_named
sys.path.remove('Tests/temp/1')

class Test(unittest.TestCase):
    def test_case1(self):
            self.assertEqual(Generic_named.main(), "Hello, World!")

if __name__ == '__main__':
    unittest.main()



cat Tests/test2.py
import unittest
import os
import sys
import shutil

if os.path.exists('Tests/temp/2'):
    shutil.rmtree('Tests/temp/2')

shutil.copytree('Tests/mock/2', 'Tests/temp/2')
shutil.copyfile('Programs/program2/Generic_named.py', 'Tests/temp/2/Generic_named.py')

sys.path.append('Tests/temp/2')
import Generic_named
sys.path.remove('Tests/temp/2')

class Test(unittest.TestCase):
    def test_case1(self):
            self.assertEqual(Generic_named.main(), "Goodbye, World!")

if __name__ == '__main__':
    unittest.main()

Both tests pass when run individually:

python3 -m unittest Tests/test1.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK


python3 -m unittest Tests/test2.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

However, they fail when being run together:

python3 -m unittest discover -p test*.py -s Tests/
.F
======================================================================
FAIL: test_case1 (test2.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/s/Documents/Coding practice/2024/Test Mess/1/Tests/test2.py", line 18, in test_case1
    self.assertEqual(Generic_named.main(), "Goodbye, World!")
AssertionError: 'Hello, World!' != 'Goodbye, World!'
- Hello, World!
+ Goodbye, World!


----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

If I try to use a different temporary name for one of the scripts I am trying to test,

cat Tests/test2.py
import unittest
import os
import sys
import shutil

if os.path.exists('Tests/temp/2'):
    shutil.rmtree('Tests/temp/2')

shutil.copytree('Tests/mock/2', 'Tests/temp/2')
shutil.copyfile('Programs/program2/Generic_named.py', 'Tests/temp/2/Generic_named1.py')

sys.path.append('Tests/temp/2')
import Generic_named1
sys.path.remove('Tests/temp/2')

class Test(unittest.TestCase):
    def test_case1(self):
            self.assertEqual(Generic_named1.main(), "Goodbye, World!")

if __name__ == '__main__':
    unittest.main()

Then I get a different error:

python3 -m unittest discover -p test*.py -s Tests/
.E
======================================================================
ERROR: test_case1 (test2.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/s/Documents/Coding practice/2024/Test Mess/2/Tests/test2.py", line 18, in test_case1
    self.assertEqual(Generic_named1.main(), "Goodbye, World!")
  File "/home/s/Documents/Coding practice/2024/Test Mess/2/Tests/temp/2/Generic_named1.py", line 4, in main
    return custom_module.out("Goodbye, World!")
TypeError: out() takes 0 positional arguments but 1 was given

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (errors=1)

It seems to be trying to import the same file, despite me using a different file from a different path with the same name. This seems strange, as I’ve been making sure to undo any changes to the Python Path after importing what I wish to test. Is there any way to mock the path? I can’t change the name of the custom_module, as that would require changing the programs I wish to test.

How should I write, approach, or setup these tests such that they can be tested with unittest discover the same as they can individually?