ninjaturtles Issue Tracker Rss Feed

Created Issue: Console runner doesn't recognise nested classes [5]

davidmus — Mon, 20 Aug 2012 08:33:47 GMT

For example, the following command line fails with an "Unknown type" message:

NinjaTurtles.Console run -c NinjaTurtles.MutationTest.MergeReportingStrategy NinjaTurtles.Tests.dll

Commented Issue: Mutate uncompiled code [4]

davidmus — Thu, 16 Aug 2012 08:39:14 GMT

Hey guys, I saw your answer to this question on StackOverFlow: http://stackoverflow.com/questions/246495/what-mutation-testing-frameworks-exist

I was looking for something that would changed the implementation code, and run my existing test suite (as opposed to having to write specific mutation based tests). If the mutated code causes a failure in my test suite, than that's a good thing (it means that the mutation didn't go unnoticed).

Thoughts? Could I use part of your project to work through this problem area?
Comments: We've just added a console runner that allows you to run mutation testing over an existing code base and test suite without writing specific mutation testing. It's still operating at the IL, rather than the uncompiled code, level, but it might help you?

Commented Issue: Mutate uncompiled code [4]

davidmus — Mon, 06 Aug 2012 07:52:50 GMT

Hey guys, I saw your answer to this question on StackOverFlow: http://stackoverflow.com/questions/246495/what-mutation-testing-frameworks-exist

I was looking for something that would changed the implementation code, and run my existing test suite (as opposed to having to write specific mutation based tests). If the mutated code causes a failure in my test suite, than that's a good thing (it means that the mutation didn't go unnoticed).

Thoughts? Could I use part of your project to work through this problem area?
Comments: Slightly confused by this. The mutation tests you write are just to direct NinjaTurtles to run your existing test suite having changed the implementation code at the IL level. You don't exactly write "specific mutation based tests". Is your requirement just to do what NinjaTurtles does, but with the mutations applied at the source code level? This is a *much* harder problem than mutating at the IL level, and we are not convinced that it offers anything significantly more useful than IL mutation, hence our approach.

Created Issue: Mutate uncompiled code [4]

amirrajan — Fri, 03 Aug 2012 01:49:56 GMT

Commented Issue: Unit tests in nested classes are not discovered [2]

davidmus — Wed, 04 Jul 2012 14:25:23 GMT

If unit tests are within nested classes, they are not discovered and result in a MutationTestFailureException exception with the following message: No matching tests were found to run.

The following simple structure produces the exception:

UnitTestFile.cs

public class ParentTestClass
{
[TestFixture]
public class ChildTestClass
{
[Test]
public void MethodUnderTest_Tests()
{
// code to test MethodUnderTest method in ClassUnderTest class.
}

[Test, MutationTest]
public void MethodUnderTest_Mutation_Tests()
{
MutationTestBuilder<ClassUnderTest>
.For("MethodUnderTest")
.Run();
}
}
}
Comments: This should now be fine in 0.3.0.1, just released - can you check and confirm please? Thanks again.

Commented Feature: Pre-run passing test suite [1]

davidmus — Wed, 04 Jul 2012 14:13:14 GMT

Running the passing test suite against non-mutated code will give two benefits:

1. If they don't pass, there's no point running the mutation tests - just fail quickly.

2. We can establish a time benchmark for the test suite, so that mutation test runs can be timed out based on a multiple of the benchmark time rather than the current hard-coded time.
Comments: Passing mutation tests are relatively quick in most cases. This was vastly slowing the overall process and needs rethinking.

Edited Feature: Pre-run passing test suite [1]

davidmus — Wed, 04 Jul 2012 14:13:14 GMT

Edited Feature: Pre-run passing test suite [1]

davidmus — Mon, 02 Jul 2012 09:11:54 GMT

Edited Issue: Unit tests in nested classes are not discovered [2]

davidmus — Mon, 02 Jul 2012 08:52:22 GMT

Edited Issue: Unit tests in nested classes are not discovered [2]

davidmus — Mon, 02 Jul 2012 08:32:48 GMT

Created Issue: BitwiseOperatorTurtle produces equivalent mutants when applied to bit mask combinations [3]

davidmus — Mon, 02 Jul 2012 07:52:32 GMT

If the bitwise operators | or ^ are used to combine bit mask values, then this turtle produces equivalent mutants, as the two (and indeed +) are interchangeable. We need a nice way of catering for this case.

Edited Feature: Pre-run passing test suite [1]

davidmus — Sun, 01 Jul 2012 07:04:35 GMT

Edited Issue: Unit tests in nested classes are not discovered [2]

davidmus — Sun, 01 Jul 2012 07:04:20 GMT

Commented Issue: Unit tests in nested classes are not discovered [2]

davidmus — Sun, 01 Jul 2012 07:03:55 GMT

If unit tests are within nested classes, they are not discovered and result in a MutationTestFailureException exception with the following message: No matching tests were found to run.

The following simple structure produces the exception:

UnitTestFile.cs

public class ParentTestClass
{
[TestFixture]
public class ChildTestClass
{
[Test]
public void MethodUnderTest_Tests()
{
// code to test MethodUnderTest method in ClassUnderTest class.
}

[Test, MutationTest]
public void MethodUnderTest_Mutation_Tests()
{
MutationTestBuilder<ClassUnderTest>
.For("MethodUnderTest")
.Run();
}
}
}
Comments: Thanks belairb - good spot. This seems relatively easy to fix, and I'll try to get it into the next release.

Created Issue: Unit tests in nested classes are not discovered [2]

belairb — Sat, 30 Jun 2012 19:13:26 GMT

Created Feature: Pre-run passing test suite [1]

davidmus — Sat, 30 Jun 2012 14:22:13 GMT