Genstat Quality Assurance
If you prefer you can read this as a pdf. At VSN International, we recognize that your analyses depend on the accuracy and validity of our software. We therefore take great care to check that the results that we produce are reliable and correct. This document summarizes some of the methods that we use to achieve this.
We use algorithms that are designed to give numerical accuracy and stability. These are mainly produced in-house by our team of well-trained developers. We also make use of the renowned and carefully-validated algorithms produced by the Numerical Algorithms Group (one of our shareholders).
We check the Genstat directives using an extensive suite of test programs whose results have been checked by
- reproducing analyses and comparing the results for representative data sets using alternative methods e.g. anova and regression, reml and anova, etc.
- by explicit calculation, using Genstat’s ability to perform general matrix and vector calculations
- by hand calculation where feasible
- comparing our results with those in published books and refereed journal papers
- by comparing with the results from other software systems
Procedures in the Genstat Procedure Library are assessed by an Editorial Board, who use similar methods to check that accepted procedures are useful, reliable and accompanied by clear documentation.
- Our technical staff have considerable experience of software development, dating back to the early 1970’s.
- They all have degree and post-graduate qualifications in statistics or computing.
- They enjoy close collaborative links with research statisticians at organizations including Rothamsted Research and New South Wales Department of Primary Industry.
- They also operate as members of the research community, attending and presenting papers at scientific meetings and conferences; this helps to ensure that they are up-to-date with current statistical research.
- All our staff subscribe to the concept of self-development to enhance our skills to the benefit of VSNi’s activities, and VSNi has recognized the importance of this activity by becoming certified under the UK Government’s Investors in People scheme.
- The underlying material for our software (source code, test suites, documentation etc) is held in a secure archive that uses the source-control system Subversion. This keeps a formal record of every change, with explanatory documentation to explain why it was made, and allows us to step back accurately and reliably to earlier versions to facilitate the diagnosis and correction of errors.
- To guard against mistakes, our extensive suite of test programs is run after every substantive change.
- Each new release undergoes a period of at least 2 months of beta-testing by committed users to check its reliability under “field conditions”. A formal reporting process is followed to ensure that errors are logged and corrected.
- Flaws and errors subsequently reported in the full release are also logged in a formal system, and patches and service packs are produced in a timely way to correct them.
- We should also place on record the fact that many of the core Genstat algorithms have been in intensive use for many years by a wide population of very disparate users. You can therefore base your confidence in Genstat, not only on our efforts but also on the accumulated efforts of your peers.
We recognize the crucial safety-net that timely, well-informed and helpful support can provide for our supported users.
- We handle all our support calls in-house: you will not be sent ill-informed replies from an off-shore call center.
- We enter support enquiries into a monitoring system to ensure that they do not become lost or forgotten.
- Our target, achieved with over 99% success, is to provide a reply within one working day.
- Our technical staff are fully involved in support, so you are guaranteed a well-informed answer. Their involvement also ensures that flaws that are reported through support receive appropriate priority in our development. Where a flaw can be corrected quickly, a software patch will be formed. Alternatively, we endeavour to provide a work-around.
- We are happy to provide statistical advice and assistance as well as computing support. If your request is too complicated for a quick response, we will endeavour to offer you an acceptable consulting contract.
- In addition to our formal support service we also maintain web-based discussion listsfor Genstat and ASreml on the VSNi website. Although, these are intended mainly to provide a forum for users’ views and self-help, we do monitor the discussions so that we can answer any more-challenging queries.
Good statistical practice
Our technical development team includes statisticians with considerable experience of statistical consulting and teaching. We are thus very aware of the issues that can arise when statistical software is used by people with limited statistical experience or training. The menus, interfaced and algorithms in Genstat are designed to help these users to use statistics effectively and correctly. For example
- the menus for more sophisticated methods, such as generalized linear models, are designed to build naturally on more familiar methods, such as linear regression, to ease your transition into more advanced techniques
- we make a clear distinction between categorical variables (i.e. factors) and continuous numerical variables (variates) to help you to avoid using these in inappropriate ways
- the output window supports context-sensitive help to explain unfamiliar terms
- we use general algorithms, for example in our facilities in Genstat for anova, regression and reml, to avoid artificial constraints on the size of complexity of your analyses
- Genstat has a simple but powerful command language that allows you to access its full range of methods
- we have clear documentation and on-line help, and an extensive set of examples of the commands that are accessible through a specially designed menu
- the Genstat command language can be used as a general-purpose programming language so that you can develop or modify methods to match your own requirements
- development of new or customized methods is facilitated by the fact that the Genstat Procedure Library is open-source, with a menu to access the source code of any of the Library procedures
- We are not willing to provide questionable types of analysis even though some of these may have achieved misguided popularity elsewhere!
We recognize the importance of keeping you up-to-date with new developments in statistics and computing. We thus have a regular schedule of new releases. Genstat itself has for many years followed an annual release schedule. Its new Editions generally include not only new statistical methodology but also enhanced interfaces, new documentation and updates to cater for any recent new releases of MS Windows.