Procedural File: odi.library.php

 <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
 <>          ___  ___  __ _                                        <>
 <>         / _ \/ __|/ _` |               european space agency   <>
 <>        |  __/\__ \ (_| |                                       <>
 <>         \___/\___/\__,_|            agence spatiale europeenne <>
 <>                                                                <>
 <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

 European Space Agency ESA/ESTEC
 Contracts Department
 Postbus 299
 2200 AG Noorwijk
 The Netherlands

 WEB:         http://www.esa.int/
 Phone:         +31 71 565-6565

 Peter Wintoft, Swedish Institute of Space Physics and
 Daniel Heynderickx, DH Consultancy

array cdfepoch_to_date( double $cdf_epoch)

Convert CDF Epoch to UTC date string of the form YYYY-MM-DD hh:mm:ss

Tags:

Parameters

double

$cdf_epoch

The CDF EPOCH

int checkTable( mysqli $link, string $tablename, resource $lhandle)

Check if dataset is listed in table dataset and return the dataset_id

Tags:

Parameters

mysqli	$link	A MySQL database connection
string	$tablename	The ODI name of the dataset
resource	$lhandle	A file pointer to the log file.

array clean_skeleton( string $sk_file_name)

Read and clean the skeleton file to enable further parsing.

Tags:

Parameters

string

$sk_file_name

The skeleton file name.

array connect_to_database_array( array $ca)

Connect to the database using an array with connection parameters.

Tags:

Parameters

array

$ca

An array with connection parameters (see get_conn_array)

string create_column_name( string $name, array $num, string $type)

Create a column name with data type to be used in creating dataset_* table.

Tags:

Parameters

string	$name	A string with the base part of the name.
array	$num	An array with numbers.
string	$type	The MySQL data type.

void create_dataset_table( mysqli $conn, int $dataset_id, string $dataset_name)

Create a "dataset_*" table.

The column names are constructed from "data" (CDF VAR_TYPE="data") variable names and the NRV index if it is larger than 0. The tables always have the columns cdf_epoch, epoch, millisec, and dataset_file_id. Updated version with improved query syntax.

Parameters

mysqli	$conn	A MySQL database connection.
int	$dataset_id	The id of a selected dataset.
string	$dataset_name	The name of the table holding the data.

void dataset_update( mysqli $link, string $tablename, int $tableid)

Update table dataset with start and end epoch and modification date.

Parameters

mysqli	$link	A MySQL database connection
string	$tablename	The name of the dataset_* table.
int	$tableid	The id in dataset for the dataset_* table.

double date_to_cdfepoch( string $date, $msec, double $millisec)

A function to convert a date into CDF EPOCH.

Tags:

Parameters

string	$date	The date in YYYY-MM-DD hh:mm:ss format
double	$millisec	The millisecond part.
	$msec

void delete_dataset( mysqli $conn, string $dataset_name)

Delete the "dataset_*" table together with all related entries.

The function first drops the table holding the data (i.e. the dataset_* table). Then the entry in the dataset table is deleted and all associated entries in the other tables are automatically deleted.

Parameters

mysqli	$conn	A MySQL database connection.
string	$dataset_name	The name of the table holding the data.

void dirscan( string $basedir, string $filepattern)

Recursive directory scan.

Parameters

string	$basedir	The base directory to scan.
string	$filepattern	The pattern to match against the file names.

bool exists_dataset( mysqli $conn, string $dataset_name)

Check if dataset name exists in dataset table.

Tags:

Parameters

mysqli	$conn	A MySQL database connection.
string	$dataset_name	The name of the table holding the data.

int getFiles( mysqli $link, array $dfiles, array $tfiles, resource $lHandle, string $tablename, int $tableid, string $platform, string $instrument, string $sk_file, $parser)

Load temporary files load.tmp into dataset_* and update dataset and dataset_files.

Tags:

Parameters

mysqli	$link	A MySQL database connection
array	$dfiles	Multi-dim array with file names, dates, and complete path to file.
array	$tfiles	Multi-dim array with id, file names, and dates of file already ingested.
resource	$lHandle	A file pointer to the log file.
string	$tablename	The name of the dataset_* table.
int	$tableid	The id of the dataset.
string	$platform	The platform name.
string	$instrument	The instrument name.
string	$sk_file	The name of the skeleton file
	$parser

array get_cl_arguments( array $argv, array $arg_array)

A support function to get the command line arguments when a script is executed on the command line.

The first argument ($argv) is an array with the actual arguments given on the command line, including the script name itself.

The second argument ($arg_array) is an array where the values will become the keys to the returned array.

Example:

Executing a script on command line like

1.     show_metadata.php DST

uses

1.     $arg_array = array("dataset");
2.     $args = get_cl_arguments($argv,$arg_array);

where the array $args becomes

1.     $args = array ( "dataset" => "DST" )

Parameters

array	$argv
array	$arg_array

array get_conn_array( integer $user_level)

Connection parameters for a given user level.

An array is returned with the ODI database parameters based on the environment variables and the user level.

Tags:

Parameters

integer

$user_level

The user level

array get_data_and_fields( mysqli $conn, string $dataset, double $epoch_start_cdf, double $epoch_end_cdf, int $max_num_rec, array $meta)

Get the data and field names for a set of variables for a dataset.

Tags:

Parameters

mysqli	$conn	A MySQL connection.
string	$dataset	The SEDAT name of the dataset.
double	$epoch_start_cdf	Start epoch in CDF epoch format.
double	$epoch_end_cdf	End epoch in CDF epoch format.
int	$max_num_rec	Maximum number of records to be read.
array	$meta	An array with variable metadata (name,dims,sizes).

array get_nrv_values( mysqli $conn, string $dataset, string $variable_name)

Get the NRV values for a variable.

If the LABL_PTR_? attribute is not defined the returned values will just be an array of number.

Tags:

Parameters

mysqli	$conn	A MySQL database connection
string	$dataset	The ODI name of the dataset
string	$variable_name	The variable name

array get_variable_metadata( mysqli $conn, string $dataset, array $var)

Get the metadata for a list of variables.

Tags:

Parameters

mysqli	$conn	A MySQL connection.
string	$dataset	The SEDAT name of the dataset.
array	$var	An array with variable names.

int ingest_data( mysqli $link, string $tablename, $basedir, string $platform, string $instrument, string $filepattern, string $sk_file, $scanner, $parser)

Construct a list of files to be ingested, convert them to temporary file and ingest into the database.

Tags:

Parameters

mysqli	$link	A MySQL database connection
string	$tablename	The name of the dataset_* table.
string	$platform	The platform name.
string	$instrument	The instrument name.
string	$filepattern	The file pattern to match against the raw data files.
string	$sk_file	The name of the skeleton file
	$basedir
	$scanner
	$parser

void insert_dataset_entry( mysqli $conn, string $dataset_name)

Insert a row in the dataset table.

Parameters

mysqli	$conn	A MySQL database connection.
string	$dataset_name	The name of the table holding the data.

void insert_dataset_metadata_entry( mysqli $conn, array $meta)

Insert a row in the dataset_metadata table.

The

1. $meta

is an array with the following keys:

1.  "dataset", "attribute", "entry_number", "value", "data_type"

Parameters

mysqli	$conn	A MySQL database connection.
array	$meta	The name of the table holding the data.

void insert_variable_attribute_entry( mysqli $conn, array $metadata, int $dataset_id)

Insert a row in the variable_attribute table.

The variable attributes are in the array

1. $metadata

and has the following keys:

1.  "attribute"
2.  "value"
3.  "variable_metadata"
4.  "data_type"

Parameters

mysqli	$conn	A MySQL database connection.
array	$metadata	The variable attributes.
int	$dataset_id	The dataset id.

void insert_variable_metadata_entry( mysqli $conn, $metadata, array $metdata)

Insert a row in the variable_metadata table.

The variable metadata are in the array

1. $metadata

and has the following keys:

1.  "name"
2.  "dataset"
3.  "number_elements"
4.  "dims"
5.  "sizes"
6.  "record_variance"
7.  "dimension_variances"
8.  "data_type"

Parameters

mysqli	$conn	A MySQL database connection.
array	$metdata	The variable metadata.
	$metadata

void insert_variable_nrv_entry( mysqli $conn, string $variable_name, int $nrv_index, string $nrv_pos, string $nrv_value, string $dataset_id)

Insert a row in the nrv_value table.

Parameters

mysqli	$conn	A MySQL database connection.
string	$variable_name	The name of the variable.
int	$nrv_index	The index of an element in the NRV variable.
string	$nrv_pos	The positional index as given in the skeleton file.
string	$nrv_value	The value of the NRV variable at the given index.
string	$dataset_id	The id of the dataset.

string no_quote( string $s)

A function to remove quotes (").

Tags:

Parameters

string

$s

A string, with or without quotes

array odi_read_data( string $dataset, string $epoch_start, string $epoch_end, int $max_num_rec, string $var, string $var,...)

A function for reading ODI data into an array.

Tags:

Parameters

string	$dataset	The SEDAT name of the dataset.
string	$epoch_start	The start epoch in format "YYYY-MM-DD hh:mm:ss.millisec".
string	$epoch_end	The end epoch in format "YYYY-MM-DD hh:mm:ss.millisec".
int	$max_num_rec	The maximum number of records to be read.
string	$var	The name of a variable to be read.
string	$var,...	Unlimited optional variables to be read.

string quote( string $s)

A function to ensure that a string is enclosed in quotes (").

Tags:

Parameters

string

$s

A string, with or without quotes

void skt2odi( mysqli $conn, string $dataset_name, string $platform, string $platform_type, string $instrument, string $sk_file_name, string $availability, string $sedat_name, string $sedat_regions, string $sedat_particles, string $sedat_emin, string $sedat_emax)

A function to create the ODI metadata from a CDF skeleton table.

The metadata are stored into the ODI tables dataset, dataset_metadata, variable_metadata, and variable_attribute.

Parameters

mysqli	$conn	A MySQL database connection.
string	$dataset_name	The name of the dataset (e.g. dataset_index_Dst).
string	$platform	The platform (e.g. index, XMM, ...).
string	$platform_type	The platform type (e.g. virtual, satellite, ...).
string	$instrument	The instrument (e.g. Dst, ERMD, ...).
string	$sk_file_name	The skeleton file name.
string	$availability	Availability: public or private.
string	$sedat_name	The SEDAT name.
string	$sedat_regions	The SEDAT region code.
string	$sedat_particles	The SEDAT particle code.
string	$sedat_emin	The SEDAT minimum energy.
string	$sedat_emax	The SEDAT maximum energy.

string s_to_us( string $s)

A function to replace non-word characters with underscore "_".

Tags:

Parameters

string

$s

A string with possible non-word characters

array tfileList( mysqli $link, int $tableid)

Produce a list of files already ingested for the dataset.

Tags:

Parameters

mysqli	$link	A MySQL database connection
int	$tableid	The dataset id

void update_dataset_entry( mysqli $conn, array $meta)

Update an entry in the dataset table.

The

1. $meta

argument is an array with the following structure:

1.    $meta = array (
2.      "attribute" => <column name in dataset>,
3.      "value"     => <the new value>,
4.      "dataset"   => <the id of the row to be updated>

Parameters

mysqli	$conn	A MySQL database connection.
array	$meta	An array with the new entry values.