In this article
Note: The full version of the Random Data Generator is a chargeable Add-On. A limited version, where the user can generate up to 50 random responses, is available for general use without the add-on.
The Random Data Generator (RDG) enables you to test a survey by automatically generating a number of test interviews. The RDG runs through the survey, selecting random responses for the questions. It runs all script code and follows any skip logic (conditions) and answer-list filtering (code masks) that may be included in the Survey, so you are able to simulate real interviews. Note that the RDG will not execute SendMail scripts and send emails. The "data" provided for Open Text and Numeric questions will be random strings of alpha-numeric characters appropriate to the question type.
Note: Authoring must be running in Test mode, the Test database must be generated and the survey status must not be set to Closed (go to Survey Status for more information), before the RDG can create the responses. If Authoring is running in Production mode then it will switch automatically to Test and a message will be displayed when the Random Data Generator is selected. If the database has not yet been generated, then it will be generated automatically as part of the procedure. If the survey status is Closed, then no test respondent data will be generated and no warning will be given.
The RDG allows you to:
- Find runtime errors in script code. Runtime errors will be generated for example if you have a script that tries to set a question with the numeric property to something other than a number, or that refers to invalid question ids when using the f function. Syntax errors in the script will be found when you create the interview files or through the "Check Script Code" functionality (go to Checking Your Code for more information).
- Identify logical errors in the Survey – for example groups of respondents who are presented with the wrong questions, questions that are never asked because of a condition error, answers that are never displayed because of errors in a code mask etc.
- Ensure validation code works as intended. Note that custom validation code will execute and log any errors, but for the RDG it will not stop the interview and redisplay the page; it will merely record the response and continue.
- Create a set of test data that resembles the data which you would expect as a result of respondents replying to a real survey. This test data can be useful when you are creating a report for a survey before "real" data has been collected, as it will allow you to view the "finished" report.
Generating Random Data
Note: The Random Data Generator (RDG) is only available for the Test database. If the Production database is active, then you must change to Test in the Database switch at the bottom of the page - before the RDG will run.
Figure 1 - Select the Test database
- Go to the Designer > Quality Control > Random Data Generator menu command - .
- Select the number of responses you wish to create.
- Click Generate.
Figure 2 - Starting the Random Data Generator
The Random Data Generator page opens - .
Note that there is a configurable limit per server installation for the number of sets of random data that can be generated in one batch job.
Figure 3 - Selecting the desired number of responses
Note that the Test database must be generated before the RDG can create the responses. If the database has not yet been generated, then it will be generated automatically as part of the procedure. If you have made changes to the survey since your last generation of the test database and interview files, or if these have not yet been generated, you can select to do this.
If the survey has more than one language, then you can select to generate the responses either all for the default language or for a language selected randomly for each response from the list of those available.
A task progress page opens and the data is created. On completion, any errors that may have been found will be listed at the bottom of the page. Assuming the responses are created, you can now create or update the BitStream files and proceed with creating your report.
Runtime Script Errors
Any runtime script errors discovered will be listed in a pane at the bottom of the screen. Runtime script errors are those errors that would cause a live interview to abort with an error - such as that shown below.
Figure 4 - Example of an internal error
If a email address is set up in "Email address to receive emails triggered by scripting errors in interview" in "Survey settings", errors such as this in production would also trigger an email with the error message to be sent.
Click on the error Description to open the question, condition or script node containing the error. If the error description is too long to be displayed in its entirety, place the mouse pointer onto the description to display the full description as a tooltip - . Below is an example of such an error:
Figure 5 - Example of a list of errors
In this example, a q is missing in the question id in the code mask. The correct expression should be f('q4').union(set('99')).
- QID - is the question id of the question with the error (or script name if the error is in a script).
- Line - tells you on which line in that script the error occurred.
- ScriptType - is the type of script it is (code mask, validation code etc.).
- Count - is the number of respondents this error occurred for during the Random Data Generation run.
Click on Link to enter the survey for the first generated respondent who received the error, to see the responses - .
Figure 6 - Example of the Link page
If you click on the filter icon towards the right end of the row, the results in Top Line Reporting or the Results tab on questions in designer will be filtered so that you only see the results of those who received the particular error - .
Figure 7 - Example of the Filter page
Once you have made the appropriate corrections, you can select Launch survey in test mode, update the existing database and generate the interview files again. You can then click on the link in the list to test one of the interviews that failed in order to verify that your correction has solved the problem.
Validation
The random data will be created according to the properties you have set on your questions. For example it will follow the settings for number of decimals, total number of digits, maximum and minimum values, ranking, required/not required etc. This means that there will never be any errors due to standard validation. However the Random Data Generator will not be able to create data according to user-defined validation defined in the validation code field on questions. That validation code will be executed, but the interview will not stop when a validation error like that occurs; it will simply record the error and proceed. You will later be able to verify the error by entering the link of the first respondent experiencing the error, investigating the test results with Rapid Results (go to About Rapid Results for more information) or looking at the test results in Survey Data Editor (go to Modifying or Removing Responses for more information).
The figure below shows an example of a user-defined validation that checks that the number of answers to a multi question does not exceed 3:
Figure 8 - Warnings from User-Defined Validation
Proportions
If you have a Survey with one or more screening questions at the beginning, the Random Data Generator may end up generating a lot of respondents that get screened, so that only a few go through the actual Survey. To prevent this from happening, you can specify the probability that different responses in the answer-list should be selected as a percentage.
In the example below, all respondents who state that they are under the age of 18 q4 are screened.
Figure 9 - Survey with Screening
To minimize the number of "Under-age" responses, you can specify a low percentage in the RDG% column of the answer list, as shown below:
Figure 10 - Setting the RdgSingle% to prevent excessive screening
Similarly, there may be certain answers you want over-represented, for example to ensure enough respondents go through a branch of a condition. In this case you can set a high RDG % value on these answers.
The RDG % numbers must be integers from 0 to 100. You are not required to specify numbers for all items in the answer list. Items without a setting will be given a value calculated from the remaining percentage, evenly distributed.
As multi questions can have several responses, there is a difference between the percentage settings for multi and single questions. For a single question the percentages will normally total 100%, but for a multi they can total to more than 100%. For this reason there are different RDG% settings for multi and single questions, and in a reusable answer list both of these columns are available - .
Figure 11 - RdgSingle% and RdgMulti% in a reusable answer list
Answer lists may be used in several questions combined with other answers or lists, and these may be dynamically filtered when running the interviews. Therefore the sum of the percentages even for single questions does not have to be 100%. When the RDG is running, the probability of selecting an answer will be calculated based on the relative portion an answer's percentage represents compared to the sum of the percentages of the answers actually displayed.
The responses will be picked so that when you view the results the percentages will match as closely as possible the percentages you have set up.
Excluding Code or Questions
There may be certain lines of code or certain questions you wish to exclude when running the Random Data Generator. Use the function IsInRdgMode() to exclude code or questions. IsInRdgMode returns true when the RDG is running and false otherwise.