Failing to get a basic test to work.

Dec 11, 2014 at 7:32 AM
Edited Dec 11, 2014 at 7:33 AM
I'm failing to get any reaction out of NinjaTurtles. I wrote a simple class plus fixture:
public class Functions
{
    public int Add(int a, int b)
    {
        return a + b;
    }
}
[TestFixture]
public class FunctionsTests
{
    [Test]
    public void TestBasicAdd()
    {
        var functions = new Functions();
        int actual = functions.Add(2, 2);

        Assert.AreEqual(4, actual);
    }

    [Test, Category("Mutation"), MutationTest]
    public void MutateTestAdd()
    {
        MutationTestBuilder<Functions>.For("Add").Run();
    }
}
ETA: The class and fixture are in different assemblies.

Now since 2*2 also equals 4 I would expect MutateTestAdd to fail. But it doesn't. It claims that all mutants are killed. So what am I doing wrong?

BTW: It already took me forever to figure out that you have to run the mutation tests via the NUnit standalone test runner. Any VS-integrated runner choked on them. And when I naively tried to run the tests in the NinjaTurtles solution via the NUnit adapter extension, they pretty much exploded and started hundreds of console hosts and nunit runner processes.
Coordinator
Dec 11, 2014 at 8:37 AM
Hi,

try making the mutation test look like this:
[Test, Category("Mutation"), MutationTest]
    public void MutateTestAdd()
    {
        MutationTestBuilder<Functions>.For("Add").With<ArithmeticOperatorTurtle>().Run();
    }
I was unaware of the runner problem. You might want to stop the nunit addin from running too many unit tests simultaneously. I had the same problem with NCrunch. I had NCrunch ignore the mutation tests, and run them manually. You could do the same with your add in. I do know that Resharper (at least in the past) has behaved well with the mutation tests.
Dec 11, 2014 at 11:35 AM
Edited Dec 12, 2014 at 10:11 AM
Unfortunately that doesn't change anything about the outcome. I did see the altered assemblies in temp\NinjaTurtles with the plus operator replaced with minus, divide etc., but still the mutants are supposed to be killed. With<ArithmeticOperatorTurtle>() just limits the type of mutations applied, right?

After looking at the source code I finally figured out that the nunit install folder needs to be in the path variable for the tests to run, so I at least got the test to run in VS add-in test runners.

ETA: It works now when I use the NUnit Adapter extension to run the tests inside Visual Studio, i.e. the mutation test fails. For some reason it passes when I run it with the R# test runner or NCrunch.