Commons:Lua/Quick tests

Automated testing

What are quick tests?

Quick tests are invoked as soon as the module is saved an their status is displayed immediately in the module navigation bar.

What is the purpose of quick tests?

Quick tests should test for stable and commonly used functionality of a module. They are not intended to be used for testing experimental, deprecated or semi-broken features.

How should a typical quick tests behave like?

A typical quick tests should not consume a lot of resources (runtime) should be small in code (at least the part inside the main module the tests belongs to) and should make it possible to find out why it failed very quickly.

Which statuses are known

• Quick tests passed
• Bug executing tests: This means tests exist but at least one of them failed.
• Error executing tests: There is a test function but invoking the test function caused an exception to be thrown
• No test API: There is no test API in the module and it is suggested to add one

What is the reasoning of adding these quick tests, why are they so incredibly useful

Unlike other community coding which is usually done on platform allowing Continues Integration (CI) and running tests before a change is merged, this isn't the case with our modules. Someone comes, wants to improve something and presses save page. And boom! Something broke. But it's only a boom if the editor is seeing the consequences immediately. And this is where quick tests come into play: Their purpose is to immediately make editors aware of issues potentially introduced with their change.

API: How to implement a quick test?

Module:QuickTest is executing these tests.

Basically you define a function in the module you want to be automatically tested and that's it. There are 2 ways to expose these function to the test API: Hidden, in a meta table or public as a member of the module. Both ways are demonstrated here:

As a public member

local p = {}
-- The function must be named runTests()
function p.runTests()
	if lorem ~= ipsum then
		mw.log( 'expected lorem to be ipsum' )
		return false
	end
	return true
end
return p

Through a meta table

This is the prefered method; if this is defined, it will be used instead of the runTests() function.

local p = {}

--[==[
This will be the meta table and allows returning "pure data"
while still employing a quick tests function, that could e.g.
validate against a schema
--]==]
setmetatable( p, {
    -- The function must be named quickTests()
    quickTests = function ()
        if lorem ~= ipsum then
	    return false, 'expected lorem to be ipsum'
	end
        return true
    end
} )
return p

--[==[ Type this in the Lua console to run these quick tests:
=getmetatable(p).quickTests() -- must display true
--]==]

If the tests should be reported to be successful and okay, this function must return true, otherwise false optionally followed by text or debugging values; you may also display more info with mw.log('message') during the test or on failure. Usually you want to create a set of test cases and then compare a hard-coded result with the actual result of running a test. As soon as one test fails, you usually return false. Whether you decide continuing to run further tests after the first failed attempt is your decision. This will make it easier to see all failed tests at once when opening the Lua console (editing mode) and track down issues more quickly.