Skip to content

Commit 806b67d

Browse files
Update test suite: Suppress output & expected warnings + improve performance (#658)
* Coarse fix for #595 * Only include namespaces for types that are included in NWB file on export (Issue #607) * Add functionality for installing extensions * Minor fixes * Update comment * Add comment + print message when extension has been installed * Update installExtension.m * Fix changed variable name * Update matnwb_createNwbInstallExtension.m Update docstring * Create listNwbTypeHierarchy.m Add utility function for listing the type hierarchy of an nwb type * Add private method for embedding specifications to file on export * Fix variable name * Update testCreateParsedType.m * Suppress output of nwbfile * Add workflow for updating nwbInstallExtension * Add option to save extension in custom location * Create InstallExtensionTest.m * Add .env file to replace env.mat * Update .gitignore * Update docstring * Change dispExtensionInfo to return info instead of displaying + add test * Reorganize code into separate functions and add tests * Minor changes to improve test coverage * add nwbInstallExtension to docs * Update update_extension_list.yml Add schedule event for workflow to update nwbInstallExtension * Update downloadExtensionRepository.m Remove local function * Update docstring for nwbInstallExtension * Fix docstring indentation in nwbInstallExtension * Add doc pages describing how to use (ndx) extensions * Fix typo * Update +tests/+unit/InstallExtensionTest.m Co-authored-by: Ben Dichter <[email protected]> * Update docs/source/pages/getting_started/using_extensions/generating_extension_api.rst Co-authored-by: Ben Dichter <[email protected]> * Add docstrings for functions to retrieve and list extension info * Fix docstring formatting/whitespace * Update listExtensions.m Add example to docstring * Move static test methods into io.internal.h5 namespace Introduce some functions that will be useful later * Update writeEmbeddedSpecifications.m Add arguments block, fix function name * Add validateEmbeddedSpecifications * Update NwbFile.m Redefine listNwbTypes method, add validation of embedded namespaces * Create listEmbeddedSpecNamespaces.m * Update nwbExportTest.m * Update test for spec/namespace embedding * Update read_indexed_column.m * Add disclaimer in deleteGroup function * Update read_indexed_column.m * Fix broken test * add test-requirement * Fix: Ensure object is group before deleting * Fix error id * Add unittests for functions in io.internal.h5 namespace * Suppress output * Remove duplicated code * Update fillClass.m Set Required "tag" for properties derived from attributes and links * Update fillClass/fillExport to add check for dependent & required properties on export * Add warnIfRequiredDependencyMissing If a required attribute depends on a non-required Dataset or Group, this method issues a warning if the dependent attribute (property) is missing when it's parent is present * Update generated types * Add default property values to generated type classes * Update warnIfPropertyAttributeNotExported to compare value with defult value before issuing warning * Add value for required property in VectorData * Add unit tests * Supress warning in test * Update tests.util.getPythonPath and remove similar code from +tests/+system/PyNWBIOTest.m * Update GenerationTest.m Add note * Suppress warnings in DataPipeTest use verifyWarning and verifyError to make code clearer Make testFilterOverride for better test granularity * Suppress output * Improve performance of if block Make check targeted to classes that have readonly prop * Remove try/catch * Use verifyWarning instead of custom warning manipulation * Updated neurodata type classes * Add unit test * Add shared fixture * Chnage name part 1/2 * Change name part2/2 * Suppress output * Add shared test fixture to tutorialtest * Update nwbtest.m * Update nwbtest.m - Use method for sorting suite by fixture - nwbClearGenerated + cleanup should be handled by test fixtures - Add option to skip generatimg coverage report * Refactor unit tests for test-schemas for better performance (#659) * Add TypesOutputFolder as property on NwbTypeGeneratorFixture class * Add abstract class for unit tests that tests test-schemas * Move test schemas to separate folder * Move test-schema Tests to tests.unit.schema namespace and subclass from tests.unit.abstract.SchemaTest * Update class names to match filenames * Replace getRandInd with builtin randi function * Reduce number of iterations in RegionViewTest * Restore file wrongly added to PR * Add Extension fixture * Refactor - slide suite sorting * Refactor misc unittests in "tests.unit" namespace (#660) * Change misc unittests from function- to class based * Change testCreateParsedType from function to class-based test suite * Fix generate types in folder set up by Shared fixture * Update dataStubTest.m * Move tutorial tests from tests.unit to tests.system.tutorial (#661) * Update nwbtest.m * Fix bug with clearing and regenerating types * Ignore generated types from test coverage * Rename ExtensionFixture to ExtensionGenerationFixture * Add docstring to ExtensionGenerationFixture * Rename ResetGeneratedTypesFixture to NwbClearGeneratedFixture * Rename NwbTypeGeneratorFixture to GenerateCoreFixture * Fix - add default input to NwbClearGeneratedFixture * Update nwbtest - Save generated reports to fixed directory * Add docs/reports to gitignore * Add .coverageignore file and local function for collecting list of files to produce code coverage for * Fix workflow for running tests (report output directory) * Update nwbExportTest.m Added comments and a better test to test for warning with ID 'NWB:validators:MissingEmbeddedNamespace' * Update branch name references from master to main * Change branch name from master to main in md and workflow files * Fix typo * Fix failing tests * Suppress more expected warnings and outputs * Add test * Update test to test different data types * Make local function getEmptyRepresentation more explicit * Remove unused code from test * Use shared fixture in WarningsTest * Update dataStubTest, include new test in test methods block * Add shared fixture to new test MustBeH5FileTest * Add abstract NwbTestCase class with utility methods that can be reused * Suppress output from various tests * Fix failing test * Remove commented code block from ObjectView class * Add utility method to NwbTestCase class * Fix bug and add test * Add "version" as property to Namespace class * Add routine for writing a function for retrieving version of generated namespace * Regenerate core with version numbers * Update generateRstForNeurodataTypeClasses.m Ignore version function when creating list of neurodata types to document * Add version specificity to cache containing map of required properties * Delete GenerateSpecificationMixin.m * Split test class setup into methods with single responsibilities * Add factory function to create NwbFile for unit testing * Clean up nwbExport test * Clean up a few more test classes * Add fixture for setting environment variables used in test suite and remove env.mat * Update PyNWBIOTest.m Add shared fixture * Fix wrong shape of data in PhotonSeriesIOTest * Update PyNWBIOTest.py --------- Co-authored-by: Ben Dichter <[email protected]>
1 parent fedd2ce commit 806b67d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

67 files changed

+1846
-1574
lines changed

+tests/+abstract/NwbTestCase.m

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
classdef (Abstract, SharedTestFixtures = {tests.fixtures.GenerateCoreFixture}) ...
2+
NwbTestCase < matlab.unittest.TestCase
3+
% NwbTestCase - Abstract class providing a shared fixture, and utility
4+
% methods for running tests dependent on generating neurodata type classes
5+
6+
methods (Access = protected)
7+
function typesOutputFolder = getTypesOutputFolder(testCase)
8+
F = testCase.getSharedTestFixtures();
9+
isMatch = arrayfun(@(x) isa(x, 'tests.fixtures.GenerateCoreFixture'), F);
10+
F = F(isMatch);
11+
12+
typesOutputFolder = F.TypesOutputFolder;
13+
end
14+
15+
function installExtension(testCase, extensionName)
16+
typesOutputFolder = testCase.getTypesOutputFolder();
17+
18+
% Use evalc to suppress output while running tests.
19+
matlabExpression = sprintf(...
20+
'nwbInstallExtension("%s", "savedir", "%s")', ...
21+
extensionName, typesOutputFolder);
22+
evalc(matlabExpression);
23+
end
24+
25+
function clearExtension(testCase, extensionName)
26+
extensionName = char(extensionName);
27+
namespaceFolderName = strrep(extensionName, '-', '_');
28+
typesOutputFolder = testCase.getTypesOutputFolder();
29+
rmdir(fullfile(typesOutputFolder, '+types', ['+', namespaceFolderName]), 's')
30+
delete(fullfile(typesOutputFolder, 'namespaces', [extensionName '.mat']))
31+
end
32+
end
33+
34+
methods (Static, Access = protected)
35+
function [nwbFile, nwbFileCleanup] = readNwbFileWithPynwb(nwbFilename)
36+
try
37+
io = py.pynwb.NWBHDF5IO(nwbFilename);
38+
nwbFile = io.read();
39+
nwbFileCleanup = onCleanup(@(x) closePyNwbObject(io));
40+
catch ME
41+
error(ME.message)
42+
end
43+
44+
function closePyNwbObject(io)
45+
io.close()
46+
end
47+
end
48+
49+
function nwbFilename = getRandomFilename()
50+
% Assumes that this method is called from a test method
51+
functionCallStackTrace = dbstack();
52+
testName = regexp(functionCallStackTrace(2).name, '\w*$', 'match', 'once');
53+
nwbFilename = sprintf('%s_%05d.nwb', testName, randi(9999));
54+
end
55+
end
56+
end

+tests/+factory/ImagingPlane.m

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
function imaging_plane = ImagingPlane(device)
2+
% ImagingPlane - Create imaging plane with values for all required properties
3+
arguments
4+
device (1,1) types.core.Device % A device is required
5+
end
6+
imaging_plane = types.core.ImagingPlane( ...
7+
'device', types.untyped.SoftLink(device), ...
8+
'excitation_lambda', 600., ...
9+
'indicator', 'GFP', ...
10+
'location', 'my favorite brain location');
11+
end

+tests/+factory/NWBFile.m

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
function nwb = NWBFile()
2+
currentTime = datetime("now", 'TimeZone', 'local');
3+
4+
nwb = NwbFile( ...
5+
'session_description', 'NWB File with required properties for unit testing', ...
6+
'identifier', 'test_file', ...
7+
'session_start_time', currentTime, ...
8+
'timestamps_reference_time', currentTime);
9+
end
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
function timeSeries = TimeSeriesWithTimestamps()
2+
timeSeries = types.core.TimeSeries(...
3+
'data', rand(1,10), ...
4+
'timestamps', 1:10, ...
5+
'data_unit', 'test');
6+
end
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
classdef ExtensionGenerationFixture < matlab.unittest.fixtures.Fixture
2+
%EXTENSIONGENERATIONFIXTURE - Fixture for generating an NWB extension.
3+
%
4+
% EXTENSIONGENERATIONFIXTURE provides a fixture for generating extension code
5+
% from an NWB specification's namespace file. When the testing framework
6+
% sets up the fixture, it calls the generateExtension function to produce the
7+
% necessary code in the specified output folder. When the framework tears down
8+
% the fixture, it removes the generated files and associated cache data,
9+
% ensuring that no artifacts remain from the test generation process.
10+
%
11+
% See also matlab.unittest.fixtures.Fixture generateExtension nwbClearGenerated
12+
13+
properties
14+
% TypesOutputFolder - Folder to output generated types for test
15+
% classes that share this fixture
16+
TypesOutputFolder (1,1) string
17+
18+
% NamespaceFilepath - Path name for extension's namespace file
19+
NamespaceFilepath (1,1) string
20+
end
21+
22+
methods
23+
function fixture = ExtensionGenerationFixture(namespaceFilepath, outputFolder)
24+
fixture.NamespaceFilepath = namespaceFilepath;
25+
fixture.TypesOutputFolder = outputFolder;
26+
end
27+
end
28+
29+
methods
30+
function setup(fixture)
31+
generateExtension(fixture.NamespaceFilepath, 'savedir', fixture.TypesOutputFolder);
32+
fixture.addTeardown(@fixture.clearGenerated)
33+
end
34+
end
35+
36+
methods (Access = protected)
37+
function tf = isCompatible(fixtureA, fixtureB)
38+
tf = strcmp(fixtureA.NamespaceFilepath, fixtureB.NamespaceFilepath) ...
39+
&& strcmp(fixtureA.TypesOutputFolder, fixtureB.TypesOutputFolder);
40+
end
41+
end
42+
43+
methods (Access = private)
44+
function clearGenerated(fixture)
45+
[~, namespaceFilename] = fileparts(fixture.NamespaceFilepath);
46+
namespaceName = extractBefore(namespaceFilename, '.');
47+
48+
generatedTypesDirectory = fullfile(fixture.TypesOutputFolder, "+types", "+"+namespaceName);
49+
rmdir(generatedTypesDirectory, 's');
50+
51+
cacheFile = fullfile(fixture.TypesOutputFolder, "namespaces", namespaceName+".mat");
52+
delete(cacheFile)
53+
end
54+
end
55+
end
+44
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
classdef GenerateCoreFixture < matlab.unittest.fixtures.Fixture
2+
% GENERATECOREFIXTURE - Fixture for creating classes for NWB types in a temporary folder.
3+
%
4+
% GENERATECOREFIXTURE provides a fixture for generating classes for neurodata
5+
% types from the from the core NWB specifications. When the testing framework
6+
% sets up the fixture, it calls the generateCore function to produce the
7+
% necessary code in a temporary output folder and add it to MATLAB's path. When
8+
% the framework tears down the fixture, it clears all the classes and deletes
9+
% the temporary folder, ensuring that no artifacts remain from the test process.
10+
%
11+
% See also matlab.unittest.fixtures.Fixture generateCore
12+
properties
13+
% TypesOutputFolder - Folder to output generated types for test
14+
% classes that share this fixture
15+
TypesOutputFolder (1,1) string
16+
end
17+
18+
methods
19+
function setup(fixture)
20+
import matlab.unittest.fixtures.PathFixture
21+
import matlab.unittest.fixtures.TemporaryFolderFixture
22+
import tests.fixtures.NwbClearGeneratedFixture
23+
24+
% Use the NwbClearGeneratedFixture to clear all generated types
25+
% from the MatNWB root directory in order to preventing path
26+
% conflicts when generating new types in a temporary directory
27+
fixture.applyFixture( NwbClearGeneratedFixture )
28+
29+
% Use a fixture to add the MatNWB folder to the search path
30+
fixture.applyFixture( PathFixture( misc.getMatnwbDir() ) );
31+
32+
% Use a fixture to create a temporary working directory
33+
F = fixture.applyFixture( TemporaryFolderFixture );
34+
35+
% Generate core types in the temporary folder and add to path
36+
generateCore('savedir', F.Folder)
37+
fixture.applyFixture( PathFixture(F.Folder) );
38+
39+
% Save the folder containing cached namespaces and NWB type classes
40+
% on the fixture object
41+
fixture.TypesOutputFolder = F.Folder;
42+
end
43+
end
44+
end
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
classdef NwbClearGeneratedFixture < matlab.unittest.fixtures.Fixture
2+
% NwbClearGeneratedFixture - Fixture for clearing generated NWB classes.
3+
%
4+
% NwbClearGeneratedFixture provides a fixture for clearing all the
5+
% generated classes for NWB types from the matnwb folder. When the fixture is
6+
% set up, all generated class files for NWB types are deleted. When the
7+
% fixture is torn down, generateCore is called to regenerate the classes for
8+
% NWB types of the latest NWB version
9+
%
10+
% See also matlab.unittest.fixtures.Fixture generateCore nwbClearGenerated
11+
12+
properties
13+
TypesOutputFolder (1,1) string {mustBeFolder} = misc.getMatnwbDir
14+
end
15+
16+
methods
17+
function fixture = NwbClearGeneratedFixture(outputFolder)
18+
arguments
19+
outputFolder (1,1) string {mustBeFolder} = misc.getMatnwbDir
20+
end
21+
fixture.TypesOutputFolder = outputFolder;
22+
end
23+
end
24+
25+
methods
26+
function setup(fixture)
27+
fixture.addTeardown( ...
28+
@() generateCore('savedir', fixture.TypesOutputFolder) )
29+
nwbClearGenerated(fixture.TypesOutputFolder)
30+
end
31+
end
32+
33+
methods (Access = protected)
34+
function tf = isCompatible(fixtureA, fixtureB)
35+
tf = strcmp(fixtureA.TypesOutputFolder, fixtureB.TypesOutputFolder);
36+
end
37+
end
38+
end

+tests/+fixtures/ResetGeneratedTypesFixture.m

-16
This file was deleted.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
classdef SetEnvironmentVariableFixture < matlab.unittest.fixtures.Fixture
2+
% UsesEnvironmentVariable Fixture for setting environment variables in tests.
3+
%
4+
% This fixture reads an environment file containing key-value pairs and
5+
% sets the corresponding system environment variables prior to executing
6+
% tests. The expected format for the environment file is:
7+
%
8+
% VARIABLE_NAME=VALUE
9+
%
10+
% Lines that are empty or start with '#' (comments) are ignored.
11+
%
12+
% The fixture first attempts to load environment variables from the file
13+
% "nwbtest.env" located in the "+tests" folder. If "nwbtest.env" is not
14+
% found, it falls back to "nwbtest.default.env". When using the default file,
15+
% the fixture only applies environment variables if they are not present
16+
% in the current list of environment variables.
17+
18+
methods
19+
function setup(fixture) %#ok<MANU>
20+
21+
applyFromFile = true;
22+
envFilePath = fullfile(misc.getMatnwbDir, '+tests', 'nwbtest.env');
23+
24+
if ~isfile(envFilePath)
25+
envFilePath = fullfile(misc.getMatnwbDir, '+tests', 'nwbtest.default.env');
26+
applyFromFile = false;
27+
end
28+
29+
if exist("loadenv", "file") == 2
30+
envVariables = loadenv(envFilePath);
31+
else
32+
envVariables = readEnvFile(envFilePath);
33+
end
34+
35+
envVariableNames = string( envVariables.keys() );
36+
if ~isrow(envVariableNames); envVariableNames = envVariableNames'; end
37+
38+
for varName = envVariableNames
39+
varValue = envVariables(varName);
40+
if ~isenv(varName)
41+
setenv(varName, varValue)
42+
elseif applyFromFile && ~isempty(char(varValue))
43+
setenv(varName, varValue)
44+
end
45+
end
46+
end
47+
end
48+
end
49+
50+
function envMap = readEnvFile(filename)
51+
% readEnvFile Reads an environment file into a containers.Map.
52+
%
53+
% envMap = readEnvFile(filename) reads the file specified by 'filename'
54+
% and returns a containers.Map where each key is a variable name and each
55+
% value is the corresponding value from the file.
56+
%
57+
% Lines starting with '#' or empty lines are ignored.
58+
59+
envMap = containers.Map;
60+
61+
fileContent = fileread(filename);
62+
lines = strsplit(fileContent, newline);
63+
64+
for i = 1:numel(lines)
65+
line = lines{i};
66+
if isempty(line) || startsWith(line, '#')
67+
continue;
68+
end
69+
70+
% Find the first occurrence of '='
71+
idx = strfind(line, '=');
72+
if isempty(idx)
73+
continue; % ignore line
74+
end
75+
76+
% Use the first '=' as the delimiter
77+
key = strtrim(line(1:idx(1)-1));
78+
value = strtrim(line(idx(1)+1:end));
79+
80+
% Insert the key-value pair into the map
81+
envMap(key) = value;
82+
end
83+
end

+tests/+sanity/GenerationTest.m

+10-6
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,21 @@
11
classdef GenerationTest < matlab.unittest.TestCase
2+
% Note: Sometimes this test does not work for the first two schema versions.
3+
% Restarting MATLAB can fix this.
4+
25
properties (MethodSetupParameter)
36
schemaVersion = listSchemaVersions()
47
end
58

69
methods (TestClassSetup)
7-
function setupClass(testCase)
10+
function setupMatNWBPathFixture(testCase)
811
import matlab.unittest.fixtures.PathFixture
9-
import tests.fixtures.ResetGeneratedTypesFixture
10-
11-
rootPath = tests.util.getProjectDirectory();
12-
testCase.applyFixture( PathFixture(rootPath) );
12+
matNwbRootPath = tests.util.getProjectDirectory();
13+
testCase.applyFixture( PathFixture(matNwbRootPath) );
14+
end
1315

14-
testCase.applyFixture( ResetGeneratedTypesFixture );
16+
function setupNwbClearGeneratedFixture(testCase)
17+
import tests.fixtures.NwbClearGeneratedFixture
18+
testCase.applyFixture( NwbClearGeneratedFixture );
1519
end
1620
end
1721

0 commit comments

Comments
 (0)