Please note: This schedule is for OpenStack Active Technical Contributors participating in the Icehouse Design Summit sessions in Hong Kong. These are working sessions to determine the roadmap of the Icehouse release and make decisions across the project. To see the full OpenStack Summit schedule, including presentations, panels and workshops, go to http://openstacksummitnovember2013.sched.org.
Tempest has a lot of negative tests but there is no shared view of how many there should be, or whether they should really be part of unit tests. There have also been discussions about fuzz testing. Here are a few things that have been discussed and from which a new consensus might emerge:
- We could have a decorator or other kind of syntax that allows a declarative way to define negative tests, but which run in the same way as existing tests. These would be easier to write and review.
- We could come up with a policy for negative test coverage and move most to unit tests.
- We could have a fuzz testing framework, possibly supported by some kind of type signature for apis to allow checking for fencepost errors rather than just slinging random arguments
