On this page I'm going to explain how the test procedure is for MapWindow GIS.
The test tool and test scripts are constantly under development, making them better every time.
For now I presume you'll be testing for MapWindow v4.8
Virtual machine
If possible it's best to create a virtual machine with only your Windows version.
I use VMWare (http://www.vmware.com/) or VirtualBox (http://www.virtualbox.org/). The latter is free and Open-Source.
Before starting make sure all old versions of MapWindow have been uninstalled and all Windows Updates have been installed.
It is also best to uninstall any .Net Framework Language packs you have installed. This makes sure the possible error messages are in English thus easier to report.
Prerequisites
If you're going to test the 32-bit version of MapWindow GIS you will need .NET Framework version 3.5 and the VC++ redistributables for VS2008.
If you're going to test the 64-bit version you need .NET Framework v4 and VC++ VS2010.
You can test the latest beta installer or you can test the weekly build. In both cases you need to install the latest beta installer of MapWindow first:
- http://tinyurl.com/mwMonthly (32-bit)
- http://tinyurl.com/mwMonthlyx64 (64-bit)
After installing MapWindow make sure it works by opening on of the sample projects.
If you want to test the weekly build you need to uninstall MapWindow again. You won't be testing the installed version, but all necessary files have been checked and if needed installed.
In the following screenshots I presume you're testing the weekly build.
SVN – Getting the latest versions
To get the latest weekly build binaries and test scripts from the MapWindow SVN repository you need TortoiseSVN (http://tortoisesvn.net/downloads.html). Download and install it.
Create these folders:
C:\dev\MapWindow4Dev\Bin (for testing weekly build only)
and
C:\dev\TestingScripts
Of course you can create these folders on a different drive as well.
You need approx. 185 MB of disk space.
When you're lucky you get all the files in one run:
But it might be possible that it takes too long for one run:
To fix this do right-click on the incomplete folder (TestingScripts in this case) and select SVN Update.
Do this until you've got all the files in the folder.
Registering MapWindow
[For testing the weekly build only]
Now you need the register the ocx in order to get your environment ready.
Use the regMapWinGIS.cmd for that. It's located in C:\dev\MapWindow4Dev\Bin
To be sure it is best to run it as an Admin.
After running this command script you should get a confirmation that the OCX was correctly registered.
You need to do this again if the ocx has some major changes.
You'll notice this because you'll get new interop dlls (AxInterop.MapWinGIS.dll and Interop.MapWinGIS.dll) when you do a SVN Update.
Open one of the projects in C:\dev\TestingScripts\ScriptData\General\MapWindow-Projects
Do some zooming and panning and if possible an Identify on one of the shapefile layers.
If this is all working you're ready to go.
Update before starting
Before starting a (new) test you should update your test scripts folder first and if you're testing the weekly build the bin folder as well.
New binaries will be published on a weekly basis and we are also still working on the test scripts.
Some test scripts are available but have no test code yet and some test scripts will be merged together to speed up the testing procedure.
Also the test tool will be improved now and then.
Test tool
It is time to launch the test tool. It is located in C:\dev\MapWindow4Dev\Bin and is called TestingTool.exe.
If you are testing the installer the TestingTool.exe will be in the folder you've installed MapWindow in, most likely <program files>/MapWindow/
Select one script (skip the BASINS and D4EM scripts, you need additional binaries for those) and run it.
How it works
The test tool will start MapWindow with the location of the script as an argument. MapWindow will execute the script when it has started up.
The script closes MapWindow again.
When all selected scripts are finished the test tool will open the default browser (IE is best in this case) and show the test results.
This XML-file is formatted with a XSD file to produce a nice looking test report.
Test results
3 states are possible:
- Successful
- Failed
- OldScript
We've been trying to catch crashes, but we haven't found a good solution yet.
Oldscript means that the test script hasn't updated yet. Only a few still exist. Those will be updated in the following weeks.
Failed means that the test failed. In the test script a scenario is run, when the outcome isn't what was expected the test fails.
More detailed information will be in C:\dev\TestingScripts a verbose log file will be created with the name TestScriptsOutput-d-M-YYYY.txt (for example TestScriptsOutput-8-3-2011.txt).
Test scripts that are generated but don't yet have a test scenario will fail as well. The verbose logging will mention 'This is a generated script. Add some code'
Successful means that the test produced the result that was expected.
In a perfect world all test scripts would return successful ;)
In two ways test results are checked.
- A value is added or changed, the data is reloaded and the value is checked. For example to test adding a field to a shapefile.
- A screenshot of the map is made, some visual changes are made, a second screenshot is created and the two screenshots are compared. In some tests the screenshots should be different in other test they should be equal.
If intermediate files are needed they are saved in C:\dev\TestingScripts\Temp in a new sub folder which has the name of the test script.
Most test scripts will clean up their files but it doesn't hurt to check this folder now and then and delete the files.
After a while you have tested all scripts and reported the failing ones and you get a new installer or a new weekly build.
You can of course run all scripts again, but it is wiser to filter first. Of course start with a SVN update.
Start with selecting the previously successful scripts.
Run them to see if the new binaries still pass these tests. If some test scripts fail this time you know something seriously is wrong.
Report this immediately to me.
Now select the previously failed scripts. Some of those should pass this time.
Also select the 'OldScript' scripts if you still have some, those should be updated shortly.
If you don't update your binaries it is enough to retest the failed scripts once in a while and also test the newly created scripts.
Known failed scripts
These scripts are known to fail. No need to report them:
|
TODO |
|
|
|
|
|
|
|
|
|
|
|
|
|
We're working on new enhancements and bug fixes in MapWindow and/or one or more of its libraries. After MapWindow v4.8 RC2 is released only bug fixes will be added.
Test scripts will also be continuously updated.
Currently we are already very happy if we test the basics of all methods and properties.
We're still working on making these test scripts.
But is is also very useful to test several methods at once using real-world case examples.
In another tutorial I explain how to write test script, but if you already have some real-world case examples send them to me (including sample data).
Contact
If you have questions or need more help you can contact me at
Dit e-mailadres is beschermd tegen spambots. U heeft JavaScript nodig om het te kunnen zien.
Use the same e-mail address to send me your list of new failed scripts (please include your test result XML file).
