Background
Emails are a vital part of doing business and are considered public records under the Public Records Act 1973. Emails enable exchange of ideas, enactment of decisions and support collaboration between an increasingly dispersed workforce. In government, some emails also provide evidence essential for accountability and should be preserved as public records into the future.
Since the late 1990s, the Victorian Government has used the Lotus Notes email application as a principal communication tool both internally and externally. Key actions and decisions of public officers are captured in the email, it is a primary repository of Victorian Government records.
In its current proprietary format and accumulated (online and Linear Tape-Open) storage volumes, access, and retrieval of emails for the purpose of analysis and evidence of decisions can be difficult, expensive and time consuming. This means its value as an information source cannot be fully realised.
This compromises the Victorian Government’s reputation for transparency and accountability and poses a risk to current administration, as well as creates potential gap in the documented memory of Victoria.
About the project
Public Record Office Victoria (PROV) is undertaking a project to develop and test solutions to appropriately capture, store, appraise and dispose of Lotus Notes email accumulations.
The project has been undertaken as a series of stages outlined below.
Stage 3, 2022-24
For stage 3, PROV is developing solutions and recommendations for current and future emails captured by government in an online environment to combat the growing volume of emails held by agencies. This process may also be applied to the backlog of non-current Lotus Notes emails held by some agencies.
The main objectives for Stage 3 are:
- to explore the application of a macro appraisal approach that prioritises agencies and key accounts within those agencies
- to automate the process of identifying non-official and low-value ephemeral emails, reducing the volume of emails using open-source tools
- development of disposal and preservation advice for government agencies to apply to current and future email, as well as non-current Lotus Notes emails stored as Lotus Notes Storage Format (NSF) files on tape.
For more information, please download our Stage 3 Project Plan Summary.
Stage 2, 2019-20
For Stage 2 a collection of PROV Lotus Notes emails (approx. 1.2 million emails) were used as a sample set to explore approaches for:
- de-duplicating emails (more than 40 per cent in the sample were duplicates)
- threading emails to preserve email conversations and reduce the overall number of records
- identifying non-public records using email header analysis and domains
- converting emails into VERS Encapsulated Objects (VEOs) .
A key outcome of Stage 2 was the confirmation that the NSF is not a sustainable format for email records. This means that the backlog of Lotus Notes emails in Victorian Government will need to be managed soon, before they become obsolete.
For more information, please download our Stage 2 summary report.
Stage 1: Proof of Concept (PoC), 2017-18
For Stage 1, PROV undertook a PoC with CenITex to test an eDiscovery tool on a sample set of 4.6 million Lotus Notes emails .
The PoC focused on disposal outcomes and included:
- conducting an initial assessment to quantify and qualify a sample email data set
- identifying duplicates within the data set (43 per cent were duplicates)
- identifying low- value/non-public records within the data set by analysing domain names
- manually reviewing results to determine accuracy.
For more information, please download our Stage 1 PoC summary report.