BitStream Files

BitStream Files are a way of storing survey data in compact files. These files are optimized for fast access, to allow aggregated tables to be generated as quickly as possible for your Reportal Report. While Forsta recommends you use BitStream Files on all surveys, the benefits will be more obvious for surveys with large data sets.

Before the BitStream Files are created for a database, when Reportal wishes to generate an aggregated table it must interrogate the SQL database directly. This can be a slow process, and it will become even slower as the database size increases.

The BitStream Files are created by initiating a task in Authoring that runs through the survey database and saves the survey data to files. Once the files have been created, you can go to the Properties page for the report and specify that BitStream Files are to be used for the current report (go to the Reportal documentation for further details).

Two different BitStream file sets can be generated; one for Rapid Results reporting and one for the Reportal report. Under most circumstances you should keep the two types separate (use each type for its intended purpose), but if for some reason you need to use the same type for both applications then you should select to use the Rapid Results file set. Rapid Results reporting and the Reportal report will then always display the same data.

Note: For surveys, you can create only two different BitStream file sets: The Reportal file set, and Rapid Results file set (which is used in Rapid Results and Survey Designer, and the Results tab/Topline report in Survey Designer). However using Panel Management rules in Professional Panels you can create several file sets, taking different cuts of the data. Then for Professional Panels surveys in Reportal, a drop-down list will include the BitStream file sets you have created in panel rules (with the names you have assigned to them there). Refer to the Reportal documentation for further details.

Generating BitStream Files

Go to the Reporting > Create Reportal BitStream Files menu command.

The page shown below opens.

Figure 1 - BitStream File generation settings

1. Choose whether you wish to generate BitStream files only on complete responses (default), or on all responses.

Note: Only select all responses (uncheck the box - see below) if you want to include the results from respondents with other statuses (incomplete, screened etc.) in your reports.

2. Click OK.

The files are generated as a task in Authoring.

Once the BitStream files have been generated for a survey, the task exists. If you later go to the Create Reportal BitStream Files menu command, the settings page will have changed to that shown below.



Figure 2 - The BitStream Files generation settings page once the task exists

This page allows you to change the task, for example to make it a recurring task, and to run the task immediately. The buttons are as follows:

• Change task setup - click to open the Task Setup page - (see below).
• Update BitStream files now - click to start the BitStream File update task immediately (the task will be placed in the queue and will be run when its turn comes).
• Force new file set - normally when the BitStream file task runs, the existing files will be updated. Check this box to delete the existing files completely and replace them with new files. Note that this only affects the data that is included in the report, not the report itself, so no other actions are necessary.
3. Click Change task setup to open the Task Setup page for the task - as shown below.

Figure 3 - BitStream Generator task parameters

Go to the Recurrence tab and click Change to access the recurrence properties (go to Updating the BitStream Files for more information).

BitStream File Update Recurrence

If you are running a live survey and wish to have the BitStream files updated at regular intervals to add new responses, you can set up a recurrence pattern on the BitStream generation task. You can for example have the files updated twice a week on Mondays and Thursdays at 2 o'clock in the morning.

As the BitStream file creation process will interrogate the SQL database to retrieve the data on the survey, you are recommended to set the task to run during periods when low activity in the survey can be expected, for example late at night.

1. To set the task to repeat at regular intervals, go to the Recurrence tab and click Change.

The layout of the tab changes - .



Figure 4 - The Recurrence Pattern dialog

2. Make the settings as required then click Save.

Note: The times the task is set to run will be "server" time - the time at the server on which your survey is running. If you are sitting in a different time zone from the server, you must bear this in mind when deciding when you wish the task to run.

The tab changes to the layout shown in the example below. The next few schedules occurrences are listed.

Figure 5 - Example of the Recurrence tab with scheduled occurrences

When running a BitStream file generation on a survey where BitStream files have already been created, the existing BitStream files will be updated if:

• Responses are added, changed or deleted.
• Questions are added or removed from the survey.
• The type of a question is changed (for example multi -> single).
• Questions or loops are moved into another loop or out of a loop.
• Codes are changed or items are added to or removed from the answer lists of grids, multis and loops.

This ensures that the calculations performed on the BitStream files will be correct with respect to the content of the entire survey database at the time of the last execution of the BitStream generation task.

Note: If you use recoded variables in your survey, ensure that the recoded variables are calculated for new responses before the BitStream generation is run.

BitStream Indexes

If a large number of reports are to be created, the database is large, and/or you often need to filter for a particular question, then you can index the BitStream files. This will flag specified rows in the database, thereby reducing search times and improving performance.

You can index the BitStream file for Interview_start, Interview_end, status, single and multi questions.

To create a BitStream Index:

1. In Authoring, go to the Reporting > BitStream Indexes menu command - .



Figure 6 - Starting BitStream Indexing

The BitStream Indexes page opens - as shown in the example below.



Figure 7 - The BitStream Indexes page

2. In the Available Questions column, select the questions you wish to create the index for, and click the >> button to move them to the Index Questions column.
3. Save the changes.
4. Create the BitStream files (go to the Reportal > Create Reportal BitStream Files menu command).

A task pane opens, indicating the status of the task. Here you can set the task to re-occur if you wish to update the file at regular intervals. On completion, the file will be indexed as specified.