Loading Your Own Data

Loading your data into Epicentre requires the database to be appropriately set up. You must have the database set up for you to use. In addition, you must have the reference data and standard values loaded. These steps are covered in the sections:

• Set Up the Environment, which explains how to compile and link the software, and set up the environment variables,
• Set Up Database, which explains the steps to set up the database, and
• Populate the Database, which explains how to populate the database with the standard values.

If the environment or the database is not appropriately set up, go to these sections to find out how to set it up.

A Pattern for Loading

I have a pattern I have developed for mapping data into Epicentre and loading it. After a discussion of the loaders, the section on Mapping your data to Epicentre will explain the pattern.

Sample Load Files

The "Pattern for Loading" section gives a detailed example of mapping data into Epicentre and building the load file. This section will give sample load files, with comments as necessary. Examples are

• Loading well logs
• Loading production information

Using POSC Loaders

POSC has supplied two generic loader programs and an implementation of POSC sql (psql) which can be used to do some updates and inserts. Most of the loading is carried out by the two generic loaders. These are refLoadLevelOne and EntityLoader. These are discussed in the Data Access and Exchange Appendix H. Below will be comments on the use of these and some examples.

refLoadLevelOne

refLoadLevelOne was originally designed to load tables of reference entities into Epicentre. It does so using Level 1 DAE calls. It has been generalized so that it is a convenient loader for any data that can be put into a tabular (spreadsheet) format.

Documentation for refLoadLevelOne is given in Appendix H of the DAE Specification. Rather than repeat the documentation, this section will refer to a set of annotated load files as an example.

Properties:

1. refLoadLevelOne loads all appropriate instances of a single table.
2. It handles data types not including frame data.
3. It handles data at the logical level. (There is a comparable loader directly into the relational tables.) This means the same load file can be used for any projection of Epicentre.

Advantages:

1. refLoadLevelOne is a convenient, tab delimited ASCII text format. It consists of a 10 line header followed by the data in a tabular form.
2. The tabular form of the data is easy for exchange.
3. The tabular form of the data can be read directly into a spreadsheet.
   Caution: MS Excel does not convert an Excel file into a proper tab delimited format.

Disadvantages:

1. refLoadLevelOne must do a lookup for every attribute that is a relationship. It does not save a handle to be reused later.
2. Selection of an instance may be a complicated path - particularly through aggregates.
3. Some data situations cannot be handled by refLoadLevelOne.
4. It cannot handle frame data.

EntityLoader

EntityLoader is more flexible than refLoadLevelOne in that it can load more data types (including frame data types), and it allows some logic into the file. The cost is that it is a much larger file, and is harder to read. My approach has always been to develop a "generator" file: a file that converts a table of information into an EntityLoader file.

EntityLoader stores one instance of data into the many tables. The example for refLoadLevelOne that is referenced above stores 10 instances of wells into (first) the well table, then (second) the well_surface_point, then (third) the pty_location_2d, then (fourth) the wellbore. The EntityLoader approach is to store the first well information into all four entities, then go to the next well, etc.

Documentation for EntityLoader is given in Appendix H of the DAE Specification. Rather than repeat the documentation, this section will refer to a set of annotated load files.

Advantages:

1. EntityLoader is more efficient, since it can save instance handles and use them later.
2. It can use logic, such as "If an instance exists, use it. Otherwise create it."
3. It handles more data types.
4. It handles frame data types (in conjunction with CDTLoader.)
5. It "documents" the full range of the mapping from some table into Epicentre.
6. It is more appropriate for complicated and variable sets of data.
   For example, a table of data for which some information is missing for some instances, and other information is missing for other instances.

Disadvantages:

1. EntityLoader takes up a lot more disk space.
2. It is not convenient to read and debug (human readability).
3. It is overkill for simple tables of data.
   Tables of data for which all instances have basically the same types of data filled in.

Using Commercial Software

There is commercial software available that can load and/or access an epicentre datastore. See the following for more information:

• Prism

Building your own Loaders

This area still under construction.

See the coming attractions area for information on downloading updates to this section

Mapping your Data into Epicentre

Following is a method used by the author (John Bobbitt) of this document. It is a personal preference for how to accomplish the tasks of mapping data into Epicentre, building load files, and eventually loading the data. It is not intended to be a specification, or even to be the best practice. It is simply an illustration (by example) of a technique that works.

The following presents the process in several steps. Each of the steps has an illustration using data from the Stratton Field. Each of the steps also has a clickable reference that goes into more detail, for those who wish to pursue things further.

The first step is to understand your data. For this example, I will use the Stratton data for wells and well picks. The type of data that I will be working with is illustrated in the EntityLoader example from an earlier section. Note that the basic information is

• A local well identifier (1 in this example).
• A location with respect to two systems:
  1. The local field coordinate system (X,Y value) and
  2. The 3d seismic survey (EW line and NS line).
• Depths in three coordinate systems:
  1. Measured depths with respect to the kelly bushing.
  2. Vertical depths with respect to sea level.
  3. Two-way seismic travel times.
  These depths are given for 10 horizons, with missing data denoted by '99999'.

The next step is to get a broad view of where this information will go in Epicentre. This is the most difficult part for those new to Epicentre. It becomes easier with practice, because the same type of data is handled the same way. The attached document goes into more detail on where this data will be placed in Epicentre.

The next step is to detail the broad structure. Coming out of this step, you should know all of the entities and their attributes that you need to instantiate. In addition, you should have a good idea of the order in which these must be instantiated. See the detailed discussion for turning the broad information into this detailed information.

The next step is to develop the "program" to actually move the data from the Table to a load file. Note that some facilities may be able to go directly from the Table to the database. At POSC, under our implementation, we go from the Table to a load file, and then run one of the general loaders to go to the database.

In our case, we will develop both ways. The first will develop an EntityLoader file for the loading. The second will develop a refLoadLevelOne set of files. The end result of either will be the same.

Sample Load Files

Loading a well log

Loading production information

This area still under construction.

See the coming attractions area for information on downloading updates to this section