Related Websites

Water and Air Monitoring and Reporting

Environmental Databases

Electronic Data Transfer: Frequently Asked Questions (FAQs)

Last Updated: March 2004

FAQs

General

  1. What is EMS?
  2. Whom do I contact to find out more about sending data electronically to EMS?
  3. How do I send my data to EMS electronically?
  4. What hardware/software do I need to submit data to EMS electronically?
  5. I'd like to transfer my data to EMS electronically. What is the best option for me to use?
  6. The EMS EDT Guidelines and Responsibilities document includes three file formats, which one should I use?
  7. Do I need any training to use EDT?
  8. How do I know my file format is correct and accepted by EMS?
  9. I submitted a file to EMS electronically, how do I know EMS received the data?
  10. Are there any charges or costs associated with sending the data to EMS?
  11. How secure is the data I am sending to EMS?
  12. Where can I get a listing of EMS codes?

EDT Correction

  1. What is the Quality Assurance (QA) Index?
  2. I received a mail message indicating there were errors in my data. Who is responsible for correcting those errors?
  3. What do I do if I discover an error in the data after it was submitted to EMS?
  4. I received a mail message indicating there were errors in my data. How do I go about finding out what those errors mean?
  5. I submitted a file to EMS via the web page. What do I do if I haven't received a QA index/error report e-mail after a hour has passed?

General

1) What is EMS?

The Environmental Monitoring System (EMS) is a data repository. The system was designed to capture data results from physical /chemical, biological and toxicological analyses performed on water, air, solid waste discharges and ambient monitoring throughout the province as well as related quality assurance data. Samples are collected by ministry staff and permittees under the Environmental Management Act and analyzed by public and private laboratories. EMS is an Oracle/Unix based application using client/server technology. This technology enables users to interact with a less complicated and more responsive GUI (Graphical User Interface) application, which provides 'point and click' functionality, as well as extensive on-line help.

The system also provides an efficient means by which analytical laboratories and permit holders can transfer monitoring data electronically into the EMS database.

back to top


2) Whom do I contact to find out more about sending data electronically to EMS?

All permittees and private laboratories wishing to submit data to EMS electronically should direct their questions to the regional ministry contact or the EMS Helpdesk. These support resources will be able to answer any questions, provide relevant background information and documentation, and/or provide immediate general assistance. Private Laboratories providing analytical services for several permittees should contact the ministry, at: Labs@Victoria1.gov.bc.ca.

back to top

3) How do I send my data to EMS electronically?

EMS supports three data file transfer options. Users submitting data electronically to EMS may choose any one of the following alternatives:

Internet Web Page
  • recommended for use by Electronic Data Providers (EDPs) submitting small to medium sized datasets. The recommended file size when using this option is 150K or less;
  • recommended for use by Electronic Data Providers (EDPs) submitting small to medium sized datasets. The recommended file size when using this option is 150K or less;
  • includes an option to submit an ASCII file to EMS using supported formats; and
  • data is loaded into EMS immediately.
FTP (File Transfer Protocol)
  • more likely to be used by EDPs transmitting larger datasets on a regular basis;
  • files are transmitted directly to an EMS directory using FTP (File Transfer Protocol);
  • can be programmed by data providers to automate data transfer on a regular basis; and
  • the data is loaded into EMS nightly.
Electronic Mail (e-mail):
  • more likely to be used by data providers submitting small datasets;
  • the ASCII file must be embedded in the e-mail message and transmitted to the EMS server; and
  • less secure than the Web Page and FTP options
back to top


4) What hardware/software do I need to submit data to EMS electronically?

All data providers submitting data to EMS electronically must have a computer with Internet access and have access to an e-mail account. This will enable EMS to communicate directly with the data provider.back to top


5) I'd like to transfer my data to EMS electronically. What is the best option for me to use?

There is not one 'best ' data transfer option to be used by all data providers. The choice of which option to choose for sending data will generally be dependent on the size and frequency of the files being transferred. Permittees who are required to submit large datasets on a regular basis may decide to use the FTP option, whereas permittees who are required to submit small datasets on an infrequent basis may decide to use the interactive Web page option. Whatever option is selected, the data provider should inform their local ministry contact of the preferred data transfer method. The ministry will then arrange to provide you with a userid and password.

back to top


6) The EMS EDT Guidelines and Responsibilities document includes three file formats, which one should I use?

All new electronic data providers must use the standard EMS file format.

Permittees who are required to report under the Pulp and Paper Single Window Data Reporting must use the revised FPP format.


7) Do I need any training to use EDT? 

The EMS electronic data transfer functionality is very simple to use and should not require any formal training. Detailed instructions on how to use the various data transfer options are included in the EMS Electronic Data Transfer Guidelines and Responsibilities document.

back to top


8) How do I know my file format is correct and accepted by EMS?

All new EDT users must use the standard EMS file format. The correct file format is detailed in the EMS Electronic Data Transfer Guidelines and Responsibilities document.

In addition, all initial file transfers for new data providers must first be performed in the EMS training environment to ensure there are no file format problems. If format problems exist, an error message indicating the problems will be sent by EMS to the data provider and the regional ministry contact. The data provider must correct any file format problems and re-send the file to the training environment. Once a file has been successfully transferred in the training environment, the data provider will be advised to transfer the file(s) to the EMS production version.

back to top


9) I submitted a file to EMS electronically, how do I know EMS received the data?

When EMS receives a data file, the system processes the file, generates a Quality Assurance (QA) index/error report and sends a copy of the report to both the data provider and the local regional ministry contact via e-mail. This report will indicate the results of the data processed. These reports will be sent as follows:

Web Page and E-Mail Data Transfer Options
  • All data submitted via the Web Page or e-mail option will be processed immediately and an e-mail containing the QA index and/or error report will be sent to the data provider and the local ministry contact by return e-mail.
FTP Option
  • Data files submitted using FTP will be processed overnight. An e-mail indicating the QA index and/or error report will be sent to the data provider and the local ministry contact by e-mail the following day.
back to top


10) Are there any charges or costs associated with sending the data to EMS? 

There are normally only minimal costs associated with sending data to EMS. Many organizations will already have all the required items. These costs are primarily associated with the initial set-up and the ongoing costs of e-mail access. Anticipated costs are outlined below:

  • Computer with modem (it anticipated that most organizations already own a suitable computer, a modem costs approximately $200)
  • E-mail access (cost will be dependent on the Internet Service Provider)

11) How secure is the data I am sending to EMS?

The EMS helpdesk will assign unique EDT userids and passwords for all data providers using the Web Page and FTP option. To arrange for a userid and password, see your local regional ministry contact or contact the EMS Helpdesk. Once assigned, passwords will not expire and will only be changed at the request of the data provider. It is the data provider's responsibility to properly safeguard the userid and password assigned to them.

Data providers using the e-mail option will not require an EMS userid and password. The e-mail account and associated e-mail password will be provided by your commercial e-mail provider.

If the data provider suspects there may be a breach of security (e.g. if they receive any inappropriate data transfer messages) they should notify their regional ministry contact immediately. Receipt of such messages could indicate a possible security breach.

back to top


12) Where can I get a listing of EMS codes?

Data providers can obtain a copy of valid EMS codes from the list of EMS Support Tables available on our Internet web site.


EDT Correction

1) What is the Quality Assurance (QA) Index?

The EMS QA Index is based on an algorithm developed to indicate a basic level of scientific confidence associated with a specific dataset. The QA Index is not intended to indicate 'good' or 'bad' data or whether permit related data is in compliance, e.g.:

  • QA index of 'C' indicates the data passed all EMS calculation and validation checks.

  • QA index of 'F' indicates the data failed one or more of EMS calculation or validation checks. An error report will be generated for all data with a QA index of 'F' indicating why the data received a value of 'F'.

EMS allows the user to submit the data for QA index calculation only. This would normally be done if users wish to review any possible errors in the dataset, before formally submitting the data to EMS. The user is informed via e-mail of the QA index value and any errors associated with the data. If corrections are necessary, the data must be corrected by the user and the file resubmitted for entry into EMS. Once entered into the EMS database, the data provider can no longer modify the data.

NOTE: files submitted to EMS with a 'QA index only' option are not actually loaded into EMS. Such files need to be subsequently re-sent for actual transfer to EMS.


2) I received a mail message indicating there were errors in my data. Who is responsible for correcting those errors?

EMS generated QA index/error reports may contain two types of messages: ERROR and WARNING messages.

If major errors are found in the dataset (e.g. mandatory fields left blank or invalid codes used), EMS will prefix the record with an ERROR message. If ERROR messages are reported in the QA index/error report, the entire file will be rejected by EMS. The data must then be corrected by the data provider and the entire file resubmitted to EMS.

Records prefixed with a WARNING message indicate that the data was accepted by EMS but that the record failed one or more systems validation checks and therefore received a "lower" QA index. Once data has been accepted by EMS, data correction is restricted to ministry staff.

NOTE: If the error report contains both ERROR and WARNING messages, the data provider will still be able to correct all of the data as EMS will have rejected the entire data file.


3) What do I do if I discover an error in the data after it was submitted to EMS?

EMS will generate QA index/and error reports after the data has been processed. A copy the QA index/error reports will be sent to data submitter and the ministry Regional contact.

  1. If major errors are found in the dataset (e.g. mandatory fields left blank or invalid codes used):

    • the entire file will be rejected by EMS. All data corrections must be made by the data provider and the corrected file resubmitted to EMS.


  2. If the data is accepted by EMS, but fails one or more of the system's validation checks (i.e. generates WARNING messages):

    • the data has been entered into EMS and can only be corrected by ministry staff. Data providers must notify the ministry contact and request the necessary changes.


  3. If all data is accepted by EMS, passes the EMS validations checks, but is subsequently found to contain errors (e.g. erroneous results):

    • Correction of EMS data is restricted to ministry staff. Data providers must notify the ministry contact and request the necessary changes.
4) I received a mail message indicating there were errors in my data. How do I go about finding out what those errors mean?

We have produced a detailed EDT Error Message Interpretation page on this site. Please review this listing for an interpretation of your error message. If the error message you received is not included on this list, contact the EMS helpdesk.


5) I submitted a file to EMS via the web page. What do I do if I haven't received a QA index/error report e-mail after a hour has passed?

Contact the EMS Helpdesk. The helpdesk will investigate the problem and advise you when the problem has been resolved.


For More Information:

Environmental Quality Branch
Ministry of Environment
Government of British Columbia

PO Box 9341
Stn Prov Govt
Victoria, British Columbia
Canada V8W 9M1