Module:UnitTests
Lua
CodeDiscussionEditHistoryLinksLink count Subpages:DocumentationTestsResultsSandboxLive code All modules
UnitTests provides a unit test facility that can be used by other scripts using require. See Wikipedia:Lua#Unit_testing for details. Following is a sample from Module:Bananas/testcases:
-- Unit tests for [[Module:Bananas]]. Click talk page to run tests.
local p = require('Module:UnitTests')
function p:test_hello()
self:preprocess_equals('Hello, world!', 'Hello, world!')
end
return p
The talk page Module talk:Bananas/testcases executes it with {{#invoke: Bananas/testcases | run_tests}}
. Test methods like test_hello above must begin with "test".
Methods
editrun_tests
editrun_tests
: Runs all tests. Normally used on talk page of unit tests.
{{#invoke:Bananas/testcases|run_tests}}
- If
differs_at=1
is specified, a column will be added showing the first character position where the expected and actual results differ.
{{#invoke:Bananas/testcases|run_tests|differs_at=1}}
The test methods below that also accept an optional options
parameter which can give processing instructions for the results. If this parameter is used, it must be a Lua table. The following fields in that table are recognized:
varying=1
: any non-nil and non-false value allows the test to be executed, and both expected and actual values are compared; but if there's a difference, we'll see the warning icon instead of the failure icon : this test is still counted as successful and not as a failure when results are different; when the results are identical, you still see the success icon . This is useful for example when checking the result of a test whose output is constantly changing (e.g. a function that returns a random value), or for comparing the actual results of a tested module with the results of an legacy template or module implementing a new better behavior that was purposely changed.nowiki=1
: any non-nil and non-false value allows displaying expected and actual results in plain text form where its content will not be preprocessed by MediaWiki. Useful for tests that generate external links or links to images or videos and other complex formats, or that include named or numeric character entities for characters (such as' '
,' '
, or' '
, or' '
) which could be equivalently encoded using plain UTF-8 (such as' '
for the non-breaking space NBSP, which may also be encoded in a Lua string constant provided for the expected value by using decimal escapes for each byte of the UTF-8 sequence, such as'\194\160'
or other Lua escapes, or directly in plain text if they are not controls, reserved in the Lua syntax for string delimiters) with the same rendering in HTML. It also allows defining the expected value string more easily for designing tests providing an explicit one.
Note that strings compared in Lua are independant of the HTML or MediaWiki parsing and independant of the UTF-8 encoding implicitly used by MediaWiki, so named or numeric character entities are considered to be distinct from plain-text encoding; but the plain-text UTF-8 encoding of string constants (when it is permitted by the syntax) is equivalent to encodings using Lua escapes (starting by\
).htmlize=1
: any non-nil and non-false value is similar tonowiki=1
but allows expected and actual results in plain wikitext form, where its content will not be preprocessed by MediaWiki, but where also the HTML tags and attributes, HTML comments, and other MediaWiki magic keywords or the MediaWiki wiki syntax for tables, lists, and images will not be postprocessed by Mediawiki, showing the wikitext exactly as it is generated. In addition, the ampersand character&
(also used in HTML characters entities) will be rendered as&
, and plaintext whitespaces will be rendered in the displayed plaintext as numeric HTML character entities (	
for TAB,
for NEWLINE, 
for the ASCII SPACE, 
for NBSP) so that different encodings in the generated output can be easily distringuished visually and that the Mediawiki postprocessing (from Wikitext to HTML) will not strip or compress these whitespaces, and will not reorder them outside embedded HTML elements (when tidying its HTML output).asciionly=1
: any non-nil and non-false value allows displaying expected and actual results with all non-ASCII characters replaced by their UTF-8 encoding rendered byte per byte in decimal with Lua escapes (like\194\160
for a plain-text non-breaking space). This does not transform HTML escapes, but this can be combined withnowiki=1
orhtmlize=1
. Useful for debugging purpose when invalid UTF-8 sequences are generated by the code to test, or to help locating in the results invisible controls and or valid Unicode codepoints forbidden in HTML, all of these would not be displayed correctly in the results.- some test methods below accept additional options.
heading
editself:heading(text)
: generates an additional row in the results table, displaying some arbitrary text spanning all columns of the table. The text may contain wiki markup. Can be useful to describe the test of the meaning of "Actual" and "Expected" columns in the generated results table. Can also be used to describe a subgroup of individual tests executed and rendered in the following rows of results.
preprocess_equals
editself:preprocess_equals(text, expected, options)
: Gives a piece of wikitext to preprocess and an expected resulting value. Scripts and templates can be invoked in the same manner they would be in a page.
self:preprocess_equals('{{#invoke:Bananas | hello}}', 'Hello, world!', {nowiki=1})
preprocess_equals_many
editpreprocess_equals_many(prefix, suffix, cases, options)
: Performs a series of preprocess_equals() calls on a set of given pairs. Automatically adds the given prefix and suffix to each text.
self:preprocess_equals_many(
'{{#invoke:BananasArgs|add|', '}}',
{
{ '2|3', '5' },
{ '-2|2', '0' },
}
)
self:preprocess_equals_many(
'{{#invoke:Coordinates/sandbox|externalLink|site=', '}}',
{
{ 'GoogleMaps|globe=Mars|lat=-14.6|lon=175.5',
'//www.google.com/mars/#lat=-14.6&lon=175.5&zoom=8'
},
{ 'GeoHack|globe=Moon|lat=0.655930|lon=23.470173|lang=en',
'//geohack.toolforge.org/geohack.php?pagename=Module_talk:Coordinates/sandbox/testcases¶ms=0.655930_N_23.470173_E_globe:Moon_&language=en'
},
}
)
self:preprocess_equals_many(
'{{#invoke:Coordinates/sandbox|GeoHack_link|lat=51.48|lon=0|lang=', '}}',
{
{ 'en',
'<span class="plainlinksneverexpand">[//geohack.toolforge.org/geohack.php?pagename=Module_talk:Coordinates/sandbox/testcases¶ms=51.48_N_0_E_globe:Earth_&language=en 51° 28′ 48″ N, 0° 00′ 00″ E]</span>'
},
{ 'ru',
'<span class="plainlinksneverexpand">[//geohack.toolforge.org/geohack.php?pagename=Module_talk:Coordinates/sandbox/testcases¶ms=51.48_N_0_E_globe:Earth_&language=ru 51° 28′ 48″ с. ш., 0° 00′ 00″ в. д.]</span>'
},
},
-- nowiki disables the MediaWiki expansion of templates, so that results also display the generated wikitexts (including wikilinks, categories, wikitables,
-- wikilists, HTML tags and attributes, magic keywords), without postprocessing them to HTML (such as transforming external links with an additional icons)
{ nowiki = 1 }
)
preprocess_equals_many_same
editpreprocess_equals_many_same(prefix, suffix, cases, expected, options)
: Performs a series of preprocess_equals() calls on a set of input cases with the same expected output. Automatically adds the given prefix and suffix to each text.
self:preprocess_equals_many(
'{{#invoke:BananasArgs|add|', '}}',
{
'2|3',
'3|2',
'10|-5',
},
'5',
-- options here are not necessary, given the format of the expected output above
{ nowiki = 1 }
)
preprocess_equals_preprocess
editself:preprocess_equals_preprocess(text, expected, options)
: Gives two pieces of wikitext to preprocess and determines if they produce the same value. Useful for comparing scripts to existing templates.
self:preprocess_equals_preprocess(
'{{#invoke:Bananas | hello}}',
'{{Hello}}',
-- nowiki doesn't prevent the template expansion of '{{Hello}}' by MediaWiki, as the expected and actual results are preprocessed,
-- but it shows the wikitext with tags and attributes generated in the MediaWiki syntax, without rendering it as normal HTML.
{ nowiki = 1 }
)
preprocess_equals_preprocess_many
editself:preprocess_equals_preprocess_many(prefix, suffix, cases, options)
: Performs a series of preprocess_equals_preprocess() calls on a set of given pairs. The prefix/suffix supplied for both arguments is added automatically. If in any case the second part is not specified, the first part will be used.
self:preprocess_equals_preprocess_many(
'{{#invoke:Foo | spellnum |', '}}',
'{{spellnum', '}}',
{
{ '2' }, -- equivalent to {'2','2'},
{ '-2', '-2.0' },
},
-- options here are not necessary, given the format of the expected outputs above
{ nowiki = 1 }
)
equals
editself:equals(name, actual, expected, options)
: Gives a computed value and the expected value, and checks if they are equal according to the == operator. Useful for testing modules that are designed to be used by other modules rather than using #invoke.
- This is intended to perform tests using calls internal in Lua, without preprocessing the input text for the actual value as if it was a MediaWiki syntax. The test actual value provided as a Lua expression.
- Now
equals()
allows comparing results independantly of their type, and even allows checking circular references in tables and to check them. - In such case, special reference values are inserted, containing the datatype name, an hash sign and an ordinal id (such as
:table#2:
for a reference to the 2nd table in the result, tables basing counted from their opening[
character in the result). - Metatables attached to tables (if they are set) may also be dumped and compared using an empty key
[]
. For that you must set the optioninclude_mt
to include metatables in the expected and actual results. - It works even when datatypes are different, between numbers, booleans, strings, tables, functions or nil.
- It also works when keys have different types, or there are integer keys out of sequence; keys in tables are sorted in a stable order (starting by integer keys in sequence 1..N, then other integers, booleans, strings, tables, references to functions, other references).
self:equals('Simple addition', 2 + 2, 4)
self:equals('Simple equality test', 2 == 2, true)
self:equals('Test returning tables',
{{2 == 2}},
{{true}},
-- The nowiki option avoids MediaWiki postprocessing of Lua tables as if it was a Mediawiki syntax to expand a template.
{nowiki=1}
)
self:equals('Test returning HTML',
mw.html.create('span'):css('display', 'none'):wikitext('dummy'):tostring(),
'<span style="display:none">dummy</span>',
-- The nowiki option avoids MediaWiki postprocessing to HTML (that would be invisible in the expected and actual values shown in the results table)
{nowiki=1}
)
equals_deep
editself:equals_deep(name, actual, expected, options)
: Like equals, but handles tables by doing a deep comparison. Neither value should contain circular references, as they are not handled by the current implementation and may result in an infinite loop.
- Legacy, now fully equivalent to
equals()
. The old restriction of use shown above no longer applies and you can compare Lua values with any type, including tables with circular references either in their keys, mapped values, or assigned metatables.
self:equals_deep('Table comparison', createRange(1,3), {1,2,3}, {nowiki=1}) -- legacy
self:equals('Table comparison', createRange(1,3), {1,2,3}, {nowiki=1}) -- now equivalent
See also
edit- Module:ScribuntoUnit – alternative unit test module
Code
-- UnitTester provides unit testing for other Lua scripts. For details see [[Wikipedia:Lua#Unit_testing]].
-- For user documentation see talk page.
local UnitTester = {}
local libraryUtil = require 'libraryUtil'
local checkType, checkTypeMulti = libraryUtil.checkType, libraryUtil.checkTypeMulti
--------------------------------------------------------------------------------------------------------------------------
local val_to_str; do
-- Cached function references (for performance).
local byte = string.byte
local find = string.find
local match = string.match
local gsub = string.gsub
local format = string.format
local insert = table.insert
local sort = table.sort
local concat = table.concat
-- For escaping string values
local str_escape_map = {
['\a'] = '\\a', ['\b'] = '\\b', ['\t'] = '\\t', ['\n'] = '\\n',
['\v'] = '\\v', ['\f'] = '\\f', ['\r'] = '\\r', ['\\'] = '\\\\' }
local str_escape_replace = function(c)
return str_escape_map[c] or format('\\%03d', byte(c))
end
-- Keys are comparable only if the same type, otherwise just sort them by type.
local types_order, ref_types_order = {
['number'] = 0, ['boolean'] = 1, ['string'] = 2, ['table'] = 3,
['function'] = 4 }, 5
function compare_keys(k1, k2)
local t1, t2 = type(k1), type(k2)
if t1 ~= t2 then -- not the same type
return (types_order[t1] or ref_types_order)
< (types_order[t2] or ref_types_order)
elseif t1 == 'number' or t1 == 'string' then -- comparing numbers (including NaNs or infinites) or strings
return k1 < k2 -- keys with the same comparable type
elseif t1 == 'boolean' then -- comparing booleans
return not k1 -- sort false before true
else -- comparing references
return tostring(k1) < tostring(k2)
end
end
-- String keys matching valid identifiers that are reserved by Lua.
local reserved_keys = {
['and'] = 1, ['break'] = 1, ['do'] = 1, ['else'] = 1,
['elseif'] = 1, ['end'] = 1, ['false'] = 1, ['for'] = 1,
['function'] = 1, ['if'] = 1, ['in'] = 1, ['local'] = 1,
['nil'] = 1, ['not'] = 1, ['or'] = 1, ['repeat'] = 1,
['return'] = 1, ['then'] = 1, ['true'] = 1, ['until'] = 1,
['while'] = 1 }
-- Main function.
val_to_str = function(val, options)
-- Decode and cache the options.
local include_mt = options and options.include_mt
local prettyprint = options and options.prettyprint
local asciionly = options and options.asciionly
-- Precompute the output formats depending on options.
local open = prettyprint and '{ ' or '{'
local equals = prettyprint and ' = ' or '='
local comma = prettyprint and ', ' or ','
local close = prettyprint and ' }' or '}'
-- What to escape: C0 controls, the backslash, and optionally non-ASCII bytes.
local str_escape_pattern = asciionly and '[%z\001-\031\\\127-\255]' or '[%z\001-\031\\\127]'
-- Indexed references (mapped to ids), and counters per ref type.
local ref_ids, ref_counts = {}, {}
-- Helper needed to detect recursive tables and avoid infinite loops.
local function visit(ref)
local typ = type(ref)
if typ == 'number' or typ == 'boolean' then
return tostring(ref)
elseif typ == 'string' then
if find(ref, "'") then
str_escape_map['"'] = '\\"'
return '"' .. gsub(ref, str_escape_pattern, str_escape_replace) .. '"'
else
str_escape_map['"'] = '"'
return "'" .. gsub(ref, str_escape_pattern, str_escape_replace) .. "'"
end
elseif typ == 'table' then
local id = ref_ids[ref]
if id then
return ':' .. typ .. '#' .. id .. ':'
end
id = (ref_counts[typ] or 0) + 1; ref_ids[ref], ref_counts[typ] = id, id
-- First dump keys that are in sequence.
local result, sequenced, keys = {}, {}, {}
for i, val in ipairs(ref) do
insert(result, visit(val))
sequenced[i] = true
end
-- Then dump other keys out of sequence, in a stable order.
for key, _ in pairs(ref) do
if not sequenced[key] then
insert(keys, key)
end
end
sequenced = nil -- Free the temp table no longer needed.
-- Sorting keys (of any type) is needed for stable comparison of results.
sort(keys, compare_keys)
for _, key in ipairs(keys) do
insert(result,
(type(key) == 'string' and
not reserved_keys[key] and match(key, '^[%a_][%d%a_]*$') and
key or '[' .. visit(key) .. ']') ..
equals .. visit(ref[key]))
end
keys = nil -- Free the temp table no longer needed.
-- Finally dump the metatable (with pseudo-key '[]'), if there's one.
if include_mt then
ref = getmetatable(ref)
if ref then
insert(result, '[]' .. equals .. visit(ref))
end
end
-- Pack the result string.
-- TODO: improve pretty-printing with newlines/indentation
return open .. concat(result, comma) .. close
elseif typ ~= 'nil' then -- other reference types (function, userdata, etc.)
local id = ref_ids[ref]
if not id then
id = (ref_counts[typ] or 0) + 1; ref_ids[ref], ref_counts[ref] = id, id
end
return ':' .. typ .. '#' .. id .. ':'
else
return 'nil'
end
end
return visit(val)
end
end
--------------------------------------------------------------------------------------------------------------------------
local htmlize; do -- For rendering valid UTF-8 HTML code (possibly multiline), as a visible plain text (on a single line that can fit in a wikitable cell)
local escaping_ascii = '[\t\n\r&<>%[%]_{|}~]' -- ASCII characters encoded on 1 byte in UTF-8, that should be displayed as HTML entities below.
local html_entities = { -- All named character entities should be valid in HTML 5.2+ (https://html.spec.whatwg.org/multipage/named-characters.html).
['\t'] = '	', -- Display whitespace controls visibly on one-line plaintext
['\n'] = ' ',
['\r'] = ' ',
['&'] = '&', -- Required here, because we use '&' for rendering all character entities in this table.
['\194\160'] = ' ', -- U+00A0 (NON-BREAKING SPACE, NBSP): code point value = 160 (UTF-8: 0xC2 0xA0).
['\194\173'] = '­', -- U+00AD (SOFT HYPHEN, SHY): code point value = 173 (UTF-8: 0xC2 0xAD).
['\226\128\128'] = ' ', -- U+2000 (EN QUAD): code point value = 8192 (UTF-8: 0xE2 0x80 0x80).
['\226\128\129'] = ' ', -- U+2001 (EM QUAD): code point value = 8193 (UTF-8: 0xE2 0x80 0x81).
['\226\128\130'] = ' ', -- U+2002 (EN SPACE): code point value = 8194 (UTF-8: 0xE2 0x80 0x82).
['\226\128\131'] = ' ', -- U+2003 (EM SPACE): code point value = 8195 (UTF-8: 0xE2 0x80 0x83).
['\226\128\132'] = ' ', -- U+2004 (THREE-PER-EM SPACE): code point value = 8196 (UTF-8: 0xE2 0x80 0x84).
['\226\128\133'] = ' ', -- U+2005 (FOUR-PER-EM SPACE): code point value = 8197 (UTF-8: 0xE2 0x80 0x85).
['\226\128\134'] = ' ', -- U+2006 (SIX-PER-EM SPACE): code point value = 8198 (UTF-8: 0xE2 0x80 0x86).
['\226\128\135'] = ' ', -- U+2007 (FIGURE SPACE, TABULAR SPACE): code point value = 8199 (UTF-8: 0xE2 0x80 0x87).
['\226\128\136'] = ' ', -- U+2008 (PUNCTUATION SPACE): code point value = 8200 (UTF-8: 0xE2 0x80 0x88).
['\226\128\137'] = ' ', -- U+2009 (THIN SPACE): code point value = 8201 (UTF-8: 0xE2 0x80 0x89).
['\226\128\138'] = ' ', -- U+200A (HAIR SPACE): code point value = 8202 (UTF-8: 0xE2 0x80 0x8A).
['\226\128\139'] = '​', -- U+200B (ZERO-WIDTH SPACE, ZWSP): code point value = 8203 (UTF-8: 0xE2 0x80 0x8B).
['\226\128\140'] = '‌', -- U+200C (ZERO-WIDTH NON-JOINER, ZWNJ): code point value = 8204 (UTF-8: 0xE2 0x80 0x8C).
['\226\128\141'] = '‍', -- U+200D (ZERO-WIDTH JOINER, ZWJ): code point value = 8205 (UTF-8: 0xE2 0x80 0x8D).
['\226\128\142'] = '‎', -- U+200E (LEFT-TO-RIGHT MARK, LRM): code point value = 8206 (UTF-8: 0xE2 0x80 0x8E).
['\226\128\143'] = '‏', -- U+200F (RIGHT-TO-LEFT MARK, RLM): code point value = 8207 (UTF-8: 0xE2 0x80 0x8F).
['\226\128\168'] = '
', -- U+2028 (LINE SEPARATOR, LSEP): code point value = 8232 (UTF-8: 0xE2 0x80 0xA8).
['\226\128\169'] = '
', -- U+2029 (PARAGRAPH SEPARATOR, PSEP): code point value = 8233 (UTF-8: 0xE2 0x80 0xA9).
['\226\128\170'] = '‪', -- U+202A (LEFT-TO-RIGHT EMBEDDING, LRE): code point value = 8234 (UTF-8: 0xE2 0x80 0xAA).
['\226\128\171'] = '‫', -- U+202B (RIGHT-TO-LEFT EMBEDDING, RLE): code point value = 8235 (UTF-8: 0xE2 0x80 0xAB).
['\226\128\172'] = '‬', -- U+202C (POP DIRECTIONAL FORMATTING, PDF): code point value = 8236 (UTF-8: 0xE2 0x80 0xAC).
['\226\128\173'] = '‭', -- U+202D (LEFT-TO-RIGHT OVERRIDE, LRO): code point value = 8237 (UTF-8: 0xE2 0x80 0xAD).
['\226\128\174'] = '‮', -- U+202E (RIGHT-TO-LEFT OVERRIDE, RLO): code point value = 8238 (UTF-8: 0xE2 0x80 0xAE).
['\226\128\175'] = ' ', -- U+202F (NARROW NON-BREAKING SPACE, NNBSP): code point value = 8239 (UTF-8: 0xE2 0x80 0xAF).
['\226\129\159'] = ' ', -- U+205F (MEDIUM MATHEMATICAL SPACE, MMSP): code point value = 8239 (UTF-8: 0xE2 0x81 0x9F).
['\226\129\160'] = '⁠', -- U+2060 (WORD JOINER, WJ): code point value = 8288 (UTF-8: 0xE2 0x81 0xA0).
['\226\129\161'] = '⁡', -- U+2061 (FUNCTION APPLICATION, FA): code point value = 8289 (UTF-8: 0xE2 0x81 0xA1).
['\226\129\162'] = '⁢', -- U+2062 (INVISIBLE TIMES): code point value = 8290 (UTF-8: 0xE2 0x81 0xA2).
['\226\129\163'] = '⁣', -- U+2063 (INVISIBLE SEPARATOR): code point value = 8291 (UTF-8: 0xE2 0x81 0xA3).
['\226\129\164'] = '⁤', -- U+2064 (INVISIBLE PLUS): code point value = 8292 (UTF-8: 0xE2 0x81 0xA4).
['\226\129\166'] = '⁦', -- U+2066 (LEFT-TO-RIGHT ISOLATE, LRI): code point value = 8294 (UTF-8: 0xE2 0x81 0xA6).
['\226\129\167'] = '⁧', -- U+2067 (RIGHT-TO-LEFT ISOLATE, RLI): code point value = 8295 (UTF-8: 0xE2 0x81 0xA7).
['\226\129\168'] = '⁨', -- U+2068 (FIRST STRONG ISOLATE, FSI): code point value = 8296 (UTF-8: 0xE2 0x81 0xA8).
['\226\129\169'] = '⁩', -- U+2069 (POP DIRECTIONAL ISOLATE, PDI): code point value = 8297 (UTF-8: 0xE2 0x81 0xA9).
['\227\128\128'] = ' ', -- U+3000 (IDEOGRAPHIC SPACE): code point value = 12288 (UTF-8: 0xE3 0x80 0x80).
['\239\187\191'] = '', -- U+FEFF (ZERO-WIDTH NON-BREAKING SPACE, ZWNSP, BYTE ORDER MARK, BOM): code point value = 65279 (UTF-8: 0xEF 0xBB 0xBF).
['\239\191\188'] = '', -- U+FFFC (OBJECT REPLACEMENT CHARACTER, ORC): code point value = 65532 (UTF-8: 0xEF 0xBF 0xBC).
['\239\191\189'] = '�', -- U+FFFD (REPLACEMENT CHARACTER, RC): code point value = 65532 (UTF-8: 0xE2 0x80 0xA8).
}
local U_FFFD = '\239\191\189' -- U+FFFD (REPLACEMENT CHARACTER)
local insert, concat = table.insert, table.concat
local function dump(s) -- For dumping invalid bytes in hexadecimal after U+FFFD (with options.invalid = 3).
local t = {}
for i = 1, #s do
insert(t, ('%02X'):format(s:byte(i)))
end
return U_FFFD .. concat(t) .. ';'
end
local function many(s) -- For replacing each invalid byte by U+FFFD (with options.invalid = 2).
return U_FFFD:rep(#s)
end
local forbidden = '[%z\001-\007\011\012\014-\031\127-\255]+' -- ASCII controls forbidden in HTML, and non-ASCII bytes.
htmlize = function(text, options)
local asciionly = options and options.asciionly -- Encode valid non-ASCII characters using multiple bytes in UTF-8 as HTML entities.
local replaceby = options and options.invalid and -- How to replace a sequence of bytes that are invalid in UTF-8 or forbidden in HTML:
( options.invalid == 0 and '' -- either discard the sequence silently (length minimized).
or options.invalid == 1 and U_FFFD -- or replace all bytes in the sequence by a single U+FFFD (length reduced)
or options.invalid == 2 and many -- or replace each byte in the sequence by U+FFFD (length preserved),
or options.invalid == 3 and dump -- or replace by U+FFFD + hexadecimal dump (length increased),
) or dump -- default replacement
return tostring(text)
:gsub(-- Split the text in pairs of (ASCII or leading or invalid bytes, trailing bytes) and filter them.
'([%z\001-\127\192-\255]*)([\128-\191]*)',
function(s, t)
local a = s:byte(-1) -- We just need to test the last leading byte before any trailing bytes.
if not(a) or a < 194 or a > 244 then -- The last leading byte is missing, ASCII or invalid in UTF-8.
-- All trailing bytes after a in t are also invalid.
return (s .. t):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities)
elseif a < 224 then -- The last valid leading byte should be followed only by 1 valid trailing byte.
-- The last non-ASCII character u (if it's valid) is encoded in UTF-8 as 2 bytes: a, b.
local u, b = s:sub(-1) .. t:sub(1, 1), t:byte(1)
return s:sub(1, -2):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities) ..
( (b and b > (a > 194 and 127 or 159) and b < 192
and (html_entities[u]
or asciionly and ('&#x%X;'):format((a - 192) * 64 + b - 128)
or u)
or ''
) .. t:sub(2):gsub(forbidden, replaceby) -- All other trailing bytes after b in t are also invalid.
or (s:sub(-1) .. t):gsub(forbidden, replaceby))
elseif a < 240 then -- The last valid leading byte should be followed only by 2 valid trailing bytes.
-- The last non-ASCII character u (if it's valid) is encoded in UTF-8 as 3 bytes: a, b, c.
local u, b, c = s:sub(-1) .. t:sub(1, 2), t:byte(1), t:byte(2)
return s:sub(1, -2):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities) ..
( (c and c > 127 and c < 192 and b > 127 and b < 192
and (html_entities[u]
or asciionly and ('&#x%X;'):format(((a - 224) * 64 + b - 128) * 64 + c - 128)
or u)
or ''
) .. t:sub(3):gsub(forbidden, replaceby) -- All other trailing bytes after d in t are also invalid.
or (s:sub(-1) .. t):gsub(forbidden, replaceby))
elseif a < 245 then -- The last valid leading byte should be followed only by 3 valid trailing bytes.
-- The last non-ASCII character u (if it's valid) is encoded in UTF-8 as 4 bytes: a, b, c, d.
local u, b, c, d = s:sub(-1) .. t:sub(1, 3), t:byte(1), t:byte(2), t:byte(3)
return s:sub(1, -2):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities) ..
( (d and d > 127 and d < 192 and c > 127 and c < 192 and b > (a < 244 and 127 or 143) and b < 192
and (html_entities[u]
or asciionly and ('&#x%X;'):format((((a - 240) * 64 + b - 128) * 64 + c - 128) * 64 + d - 128)
or u)
or ''
) .. t:sub(4):gsub(forbidden, replaceby) -- All other trailing bytes after d in t are also invalid.
or (s:sub(-1) .. t):gsub(forbidden, replaceby))
end
end
)
:gsub('^ ', ' ') -- Avoids the compression of a leading SPACE character and make it visible.
:gsub(' ', '  ') -- Avoids the compression of repeated SPACE characters and make them visible in pairs.
:gsub(' $', ' ') -- Avoids the compression of a trailing SPACE character and make it visible.
or nil -- Needed in Lua to discard the additional count of substitutions returned by a trailing call to gsub().
end
end
--------------------------------------------------------------------------------------------------------------------------
local function first_difference(a, b, options)
checkType('UnitTester:first_difference', 3, options, 'table', true)
if a == b then
return ''
elseif type(a) ~= type(b) then
return ('%s ≠ %s'):format(type(a), type(b))
elseif type(a) == 'string' then
local i, c, d, e = 1
while true do
c, d = a:byte(i) or -1, b:byte(i) or -1
e = c < d and d or c
if c ~= d or
e >= 192 and a:byte(i + 1) ~= b:byte(i + 1) or
e >= 224 and a:byte(i + 2) ~= b:byte(i + 2) or
e >= 240 and a:byte(i + 3) ~= b:byte(i + 3) then
return ('%d: %s%s ≠ %s%s'):format(i,
mw.text.nowiki(htmlize(val_to_str(a:sub(i, (c >= 240 and 3 or c >= 224 and 2 or c >= 192 and 1 or 0) + i), options))),
c >= 240 and ' (4 bytes)' or c >= 224 and ' (3 bytes)' or c >= 192 and ' (2 bytes)' or '',
mw.text.nowiki(htmlize(val_to_str(b:sub(i, (d >= 240 and 3 or d >= 224 and 2 or d >= 192 and 1 or 0) + i), options))),
d >= 240 and ' (4 bytes)' or d >= 224 and ' (3 bytes)' or d >= 192 and ' (2 bytes)' or '')
end
i = (c >= 240 and 4 or c >= 224 and 3 or c >= 192 and 2 or 1) + i
end
elseif type(a) == 'table' then
local m = #a < #b and #a or #b
for i = 1, m do
if a[i] ~= b[i] then
return ('%i: %s ≠ %s'):format(i, mw.text.nowiki(htmlize(val_to_str(a[i]), options)), mw.text.nowiki(htmlize(val_to_str(b[i]), options)))
end
end
else
return ('%s ≠ %s'):format(htmlize(val_to_str(a)), htmlize(val_to_str(b)))
end
end
--------------------------------------------------------------------------------------------------------------------------
local result_table; do
local format = string.format
result_table = {}
local meta = {
insert = function(self, ...)
local n = #self
for i = 1, select('#', ...) do
local val = select(i, ...)
if val ~= nil then
n = n + 1
self[n] = tostring(val)
end
end
end,
insert_format = function(self, ...)
self:insert(format(...))
end,
concat = table.concat,
tostring = table.concat,
}
meta.__index = meta
setmetatable(result_table, meta)
end
local function return_varargs(...)
return ...
end
--------------------------------------------------------------------------------------------
function UnitTester:heading(text)
checkType('UnitTester:heading', 1, text, 'string', false)
result_table:insert(
'|-\n!scope="colgroup" colspan="' ..
tostring(self.columns) ..
'" style="background:#FFD;color:#000;font-weight:normal;text-align:left"|' ..
text ..
'\n')
end
--------------------------------------------------------------------------------------------
-- All "preprocess" tests require that each case returns a single string.
-- As these tests are calling the Mediawiki preprocessor, they are much slower, use more resources, and may
-- fail with MediaWiki timeout errors (displaying no result at all), so they can't be too much comprehensive.
function UnitTester:preprocess_equals(text, expected, options)
checkType('UnitTester:preprocess_equals', 1, self, 'table', false)
checkType('UnitTester:preprocess_equals', 2, text, 'string', false)
checkType('UnitTester:preprocess_equals', 3, expected, 'string', false)
checkType('UnitTester:preprocess_equals', 4, options, 'table', true)
local actual = self.frame:preprocess(text)
local varying = options and options.varying
local display =
(options and options.display) and options.display or
(options and options.htmlize) and htmlize or
(options and options.nowiki) and mw.text.nowiki or
return_varargs
result_table:insert(
'|-\n||' ..
self['icon' ..
(actual == expected and
'Tick' or
varying and
'Warn' or
'Cross'
)
] ..
'||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
mw.text.nowiki(text) ..
'</bdi>||<bdi style="' ..
(type(expected) ~= 'string' and
'background:#FCC;' or
''
) ..
'border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
(type(expected) ~= 'string' and
mw.text.nowiki(val_to_str(expected)) or
display(expected)
) ..
'</bdi>||<bdi style="' ..
(type(actual) ~= 'string' and 'background:#FCC;' or '') ..
'border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
(type(actual) ~= 'string' and
mw.text.nowiki(val_to_str(actual)) or
display(actual)
) ..
'</bdi>' ..
(self.differs_at and
'||' .. ( (type(expected) ~= 'string' or type(actual) ~= 'string') and self.iconCross
or first_difference(expected, actual, options)
)
or ''
) ..
'\n')
if type(expected) ~= 'string' or
type(actual) ~= 'string' or
not varying and actual ~= expected then
self.num_failures = self.num_failures + 1
end
end
function UnitTester:preprocess_equals_preprocess(actual, expected, options, text)
checkType('UnitTester:preprocess_equals_preprocess', 1, self, 'table', false)
checkType('UnitTester:preprocess_equals_preprocess', 2, actual, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess', 3, expected, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess', 4, options, 'table', true)
checkType('UnitTester:preprocess_equals_preprocess', 4, text, 'string', true)
local text = text or actual
local errs; do
-- Protected call to the preprocessor which may fail: detect and preserve errors.
local s1, s2
s1, actual = pcall(self.frame.preprocess, self.frame, actual)
if not s1 then
actual = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
val_to_str(s1) .. ' --[=[#ERROR! ' ..
(type(actual) == 'string' and actual or val_to_str(actual)) ..
']=]</bdi>'
end
s2, expected = pcall(self.frame.preprocess, self.frame, expected)
if not s2 then
expected = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
val_to_str(s2) .. ' --[=[#ERROR! ' ..
(type(expected) == 'string' and expected or val_to_str(expected)) ..
']=]</bdi>'
end
-- If there was no processing error, check the return types (should be strings).
if not (s1 and s2) then
if type(actual) ~= 'string' then
actual = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
val_to_str(actual) .. '</bdi>'
end
if type(expected) ~= 'string' then
expected = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
val_to_str(expected) .. '</bdi>'
end
errs = true
end
end
if errs then
result_table:insert(
'|-\n||' ..
self.iconCross ..
'||<bdi style="border:1px solid #EAECF0;padding:1px;background:#F8F9FA;white-space:pre-wrap">' ..
mw.text.nowiki(text) ..
'</bdi>||<bdi style="background:#' ..
(type(expected) ~= 'string' and 'F8F9FA' or 'FFF') ..
';border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
(type(expected) ~= 'string' and mw.text.nowiki(expected) or display(expected)) ..
'</bdi>||<bdi style="background:#' ..
(type(actual) ~= 'string' and 'F8F9FA' or 'FFF') ..
'border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
(type(actual) ~= 'string' and mw.text.nowiki(actual) or display(actual))..
'</bdi>' ..
(self.differs_at and
'||' ..
self.iconCross
or ''
) ..
'\n')
self.num_failures = self.num_failures + 1
return
end
if options and options.stripmarker == true then
-- Option to ignore ANY strip marker when comparing actual to expected.
local _, stripmarker_id = expected:match('(\127[^\127]*UNIQ%-%-%l+%-)(%x+)(%-QINU[^\127]*\127)')
if stripmarker_id then
actual = actual:gsub(pattern, '%1' .. stripmarker_id .. '%3')
end
elseif options and options.templatestyles == true then
-- When module rendering has templatestyles strip markers, use ID from expected to prevent false test fail.
-- Get the strip marker id for templatestyles from expected (the reference); ignore first capture in pattern.
-- Strip marker pattern for '<templatestyles src="..." />' .
local _, stripmarker_id = expected:match('(\127[^\127]*UNIQ%-%-templatestyles%-)(%x+)(%-QINU[^\127]*\127)')
if stripmarker_id then
actual = actual:gsub(pattern, '%1' .. stripmarker_id .. '%3') -- Replace actual id with expected id; ignore second capture in pattern.
end
end
local varying = options and options.varying
local display =
(options and options.display) and options.display or
(options and options.htmlize) and htmlize or
(options and options.nowiki) and mw.text.nowiki or
return_varargs
result_table:insert(
'|-\n||' ..
self['icon' ..
(actual == expected and
'Tick' or
varying and
'Warn' or
'Cross'
)
] ..
'||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
mw.text.nowiki(text) ..
'</bdi>||<bdi style="background:#FFF;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
display(expected) ..
'</bdi>||<bdi style="background:#FFF;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
display(actual) ..
(self.differs_at and
'||' .. first_difference(expected, actual, options)
or ''
) ..
'</bdi>' ..
'\n'
)
if not varying and actual ~= expected then
self.num_failures = self.num_failures + 1
end
end
function UnitTester:preprocess_equals_many(prefix, suffix, cases, options, text)
checkType('UnitTester:preprocess_equals_many', 1, self, 'table', false)
checkType('UnitTester:preprocess_equals_many', 2, prefix, 'string', false)
checkType('UnitTester:preprocess_equals_many', 3, suffix, 'string', false)
checkType('UnitTester:preprocess_equals_many', 4, cases, 'table', false)
checkType('UnitTester:preprocess_equals_many', 5, options, 'table', true)
checkType('UnitTester:preprocess_equals_many', 6, text, 'string', true)
for _, case in ipairs(cases) do
self:preprocess_equals(prefix .. case[1] .. suffix, case[2], options, text)
end
end
function UnitTester:preprocess_equals_many_same(prefix, suffix, cases, expected, options, text)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 1, self, 'table', false)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 2, prefix, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 3, suffix, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 4, cases, 'table', false)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 5, expected, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 6, options, 'table', true)
checkType('UnitTester:preprocess_equals_preprocess_many_same', 7, text, 'string', true)
for _, case in ipairs(cases) do
self:preprocess_equals(prefix .. case .. suffix, expected, options, text)
end
end
function UnitTester:preprocess_equals_preprocess_many(prefix1, suffix1, prefix2, suffix2, cases, options, text)
checkType('UnitTester:preprocess_equals_preprocess_many', 1, self, 'table', false)
checkType('UnitTester:preprocess_equals_preprocess_many', 2, prefix1, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many', 3, suffix1, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many', 4, prefix2, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many', 5, suffix2, 'string', false)
checkType('UnitTester:preprocess_equals_preprocess_many', 6, cases, 'table', false)
checkType('UnitTester:preprocess_equals_preprocess_many', 7, options, 'table', true)
checkType('UnitTester:preprocess_equals_preprocess_many', 8, text, 'string', true)
for _, case in ipairs(cases) do
self:preprocess_equals_preprocess(prefix1 .. case[1] .. suffix1, prefix2 .. (case[2] and case[2] or case[1]) .. suffix2, options, text)
end
end
--------------------------------------------------------------------------------------------
-- All tests without "preprocess" allow each case to return any Lua type for actual and expected.
-- These tests use only Lua functions, are much faster, use less resources.
function UnitTester:equals(text, actual, expected, options)
checkType('UnitTester:equals', 1, self, 'table', false)
checkType('UnitTester:equals', 2, text, 'string', false)
checkType('UnitTester:equals', 5, options, 'table', true)
expected, actual = val_to_str(expected, options), val_to_str(actual, options)
local varying = options and options.varying
local display =
(options and options.display) and options.display or
(options and options.htmlize) and htmlize or
(options and options.nowiki) and mw.text.nowiki or
return_varargs
result_table:insert(
'|-\n||' ..
self['icon' ..
(actual == expected and 'Tick'
or varying and 'Warn'
or 'Cross'
)
] ..
'||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
mw.text.nowiki(text) ..
'</bdi>||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
mw.text.nowiki(expected) ..
'</bdi>||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
mw.text.nowiki(actual) ..
'</bdi>' ..
(self.differs_at and
'||' .. first_difference(expected, actual, options)
or ''
) ..
'\n'
)
if not varying and actual ~= expected then
self.num_failures = self.num_failures + 1
end
end
-- Legacy: now UnitTester:equals() is deep by default and properly handles tables
UnitTester.equals_deep = UnitTester.equals
--------------------------------------------------------------------------------------------
function UnitTester:iterate(cases, func)
checkType('UnitTester:iterate', 1, cases, 'table')
checkType('UnitTester:iterate', 2, func, 'function')
func = self[func]
for i, example in ipairs(cases) do
checkTypeMulti('UnitTester:iterate(cases)', i, cases, {'table', 'string'})
if type(example) == 'string' then
self:heading(example)
else
func(self, unpack(example))
end
end
end
--------------------------------------------------------------------------------------------
-- Main function that enumerates tests and run them
function UnitTester:run(frame)
self.frame = frame
self.options = frame.args.options
self.differs_at = frame.args.differs_at
-- Get the list of tests and them into alphabetical order.
local test_names = {}
for key, value in pairs(self) do
if key:find('^test') then
table.insert(test_names, key)
end
end
table.sort(test_names)
local thead_rows = 1
local thead =
'!scope="col" style="max-width:32%"|Expected\n' ..
'!scope="col" style="max-width:32%"|Actual\n'
self.columns = 2
if self.differs_at then
thead = thead ..
'!scope="col" style="width:6em"|Diff. at\n'
self.columns = self.columns + 1
end
thead =
'|-\n' ..
'!scope="col"' .. (thead_rows > 1 and ' rowspan="' .. tostring(thead_rows) .. '"' or '') .. ' style="width:32px"|\n' ..
'!scope="col"' .. (thead_rows > 1 and ' rowspan="' .. tostring(thead_rows) .. '"' or '') .. ' style="max-width:32%"|Text\n' ..
thead
self.columns = self.columns + 2
if not self.iconTick then
-- Icons are preprocessed early before running tests rather than after packing results.
-- This reduces the number of template expansions for these icons packed in results.
self.iconTick = frame:expandTemplate{ title = 'Tick', args = {} }
self.iconWarn = frame:expandTemplate{ title = 'Warn', args = {} }
self.iconCross = frame:expandTemplate{ title = 'Cross', args = {} }
end
local display_options = self.options and
'<br /><span style="font-size:smaller;font-weight:normal">' ..
'Options: <kbd>' .. val_to_str(options) ..
'</kbd></span>' or ''
self.num_failures = 0
-- Add results into the results table.
for i, test_name in ipairs(test_names) do
local caption = test_name
:gsub('^test_?([%d]+[%a]*)_(.-)$', 'Test %1: %2')
:gsub('^test_?(.-)$', 'Test: %1')
:gsub('__', ' ')
result_table:insert(
'{|class="wikitable" cellspacing="0" cellpadding="0" style="margin:.6em 0 2px;width:100%;max-width:100%;overflow-wrap:anywhere"\n' ..
'|+|' .. caption .. display_options .. '\n' ..
thead)
self[test_name](self)
--[[
local ok, result = pcall(self[test_name], self)
if not ok then
self:heading('<strong>An error occured while running this test:</strong> '.. tostring(result))
end
--]]
result_table:insert('|}\n')
end
-- Pack results.
return (self.num_failures == 0 and
'<strong style="color:#080">All tests passed.</strong>\n\n' or
'<strong style="color:#800">' .. self.num_failures .. ' tests failed.</strong>\n\n'
) .. frame:preprocess(result_table:concat())
end
function UnitTester:new()
local o = {}
setmetatable(o, self)
self.__index = self
return o
end
local p = UnitTester:new()
function p.run_tests(frame) return p:run(frame) end
-- Additional exports for tests of this module in the console.
p.val_to_str = val_to_str
p.htmlize = htmlize
p.first_difference = first_difference
return p