Guide for IDL

Installation

Untar and unzip the libebf_idl-x.x.x.tar.gz file. Then copy the folder libebf_idl-x.x.x to a location of our choice.

If the location of the folder is not in your search path, then add the libebef_idl folder to your !PATH as follows. In idl_startup.pro add the follwing lines (change ‘path_to_current_dir’ to appropriate location of libebf_idl folder)

!path=expand_path('+/path_to_current_dir/libebf_idl-x.x.x')+':'+!path

If you have an older version of the library, then

1) Remove the old libebf_idl folder.
2) Remove the old DLM file by running from IDL command prompt
IDL>remove_libebf_dlm

To check that old DLM is removed and your are using the current version

IDL>x=ebftable_get('','',version=v) & print,v
should print
file not found:
x.x.x

To run a test suite do (has to be run from libebf_idl folder)

$cd libebf_idl-x.x.x
$idl
IDL>.rnew ebf_test.pro

Examples

An interactive session explaining the usage is given below:

Generate some data.

>>> x=findgen(10) & y=findgen(10) &  z=findgen(10)

Write data to an ebf file

>>> ebf_write,'test.ebf','/data1/x1',x

Append data to an existing ebf file

>>> ebf_write,'test.ebf','/data1/y1',y,/APPEND
>>> ebf_write,'test.ebf','/data1/z1',z,/APPEND
>>> ebf_write,'test.ebf','/data1/data2/x1',x*2,/APPEND
>>> ebf_write,'test.ebf','/data1/data2/y1',y*2,/APPEND
>>> ebf_write,'test.ebf','/data1/data2/z1',z*2,/APPEND

Get a summary of the file

>>> ebf_info,'test.ebf'

Read a variable

>>> ebf_read, 'test.ebf', '/data1/z1', myvar
Data is loaded such that myvar  is  z

Write and read units

>>> ebf_read, 'test.ebf', '/data1/z1', myvar,DATAUNIT="100 km/s"
>>> ebf_read, 'test.ebf', '/data1/z1', myvar,DATAUNIT=units1
>>> print,units1

Read a variable into a structure one by one.

>>> ebf_read, 'test.ebf', '/data1/x1', myvar, /append
>>> ebf_read, 'test.ebf', '/data1/y1', myvar, /append
Data is loaded such that myvar.x1  is  x , myvar.y1 is y

Note appending to a structure in IDL is inherently a slow process as it involves a copy operation. Hash tables are generally preferable for such applications. Read a variable into a hashtable class of Craig Markwardt, one by one

>>> myvar=obj_new('hashtable')
>>> ebf_read, 'test.ebf', '/data1/x1', myvar, /append
>>> ebf_read, 'test.ebf', '/data1/y1', myvar, /append
>>> print,myvar->get('x1'),myvar->get('y1')

Calling without append will delete previous hash object and create a new one

>>> ebf_read, 'test.ebf','/data1/y1',myvar
>>> print,myvar->get('y1')
>>> obj_destroy(myvar)
As IDL versions less than 8 do not have garbage collection on heap variable,
one has to explicitly destroy the object after use.

Read a variable into an IDL8.0 hash object one by one. For IDL versions >=8 there is no need to explicitly destroy the object after use.

>>> myvar=obj_new('hashtable')
>>> ebf_read, 'test.ebf', '/data1/x1', myvar, /append
>>> ebf_read, 'test.ebf', '/data1/y1', myvar, /append
>>> print,myvar['x1'], myvar['y1']

Calling without append will delete previous hash object and create a new one

>>> ebf_read, 'test.ebf', '/data1/y1', myvar
>>> print,myvar['y1']

Read multiple variables into a structure

>>> ebf_read,'test.ebf','/data1/',mystruct
This will load '/data1/x1', '/data1/y1' and '/data1/z1' in structure
mystruct such that mystruct.x1 is x, mystruct.y1 is y  and
mystruct.z1 is z. The field names of the structures are obtained by
stripping the path from the data name. Data objects whose names
contain '.' are excluded.

Read specific items into a structure

>>> ebf_read,'test.ebf',['/data1/x1','/data1/data2/z1','/FeH'],mystruct,ALIASES=['x','z]
This loads such that  mystruct.x is x  and  mystruct.z is z*2.

Write a structure and read it back

>>> mystruct1={x:x,y:y,z:z}
>>> ebf_write,'test.ebf','/mystruct1',mystruct1
>>> ebf_read,'test.ebf','/mystruct1',mystruct11

Write a nested structure and read it back

>>> mystruct2={x:x,y:y,z:z,mystr1:mystruct1}
>>> ebf_write,'test.ebf','/mystruct1',mystruct2
>>> ebf_read,'test.ebf','/mystruct1',mystruct22

Split a structure into homogeneous arrays and write and read

>>> mystruct1={x:x,y:y,z:z}
>>> ebf_write,'test.ebf','/foo/',mystruct1
>>> ebf_read,'test.ebf','/foo/',mystruct2
Written as '/foo/x , /foo/y and /foo/z'

Write and read specific tagnames only

>>> mystruct1={x:x,y:y,z:z,p:p}
>>> ebf_write,'test.ebf','/foo/',mystruct1,tagnames=["x","z"]
>>> ebf_read,'test.ebf','/foo/'+["x","z"],mystruct2

Write and read specific tagnames only but with aliases

>>> mystruct1={x:x,y:y,z:z,p:p}
>>> ebf_write,'test.ebf','/foo/',mystruct1,tagnames=["x","z"],aliases=['x1','y1']
>>> ebf_read,'test.ebf','/foo/'+["x1","z1"],mystruct2,aliases=['x2','z2']

Read and append to a hash table instaed of a structure tabl

>>> mystruct1={x:x,y:y,z:z,p:p}
>>> ebf_write,'test.ebf','/foo/',mystruct1,tagnames=["x","z"],aliases=['x1','y1']
>>> ebf_read,'test.ebf','/foo/'+["x1","z1"],mystruct2,aliases=['x2','z2']

Routines

ebf_read

;+
; NAME:
;       EBF_READ
;
; PURPOSE:
;       This procedure reads data from an ebf-format files in the form of a
;       variable or an idl anonymous structure.
;
; DESCRIPTION
;       Data objects in an ebf file are stored like unix filenames
;       starting with / as root directory e.g.  '/FeH',  '/SDSS/g' ,
;       '/SDSS/r'. Using this analogy files correspond to data objects.
;
;
; CALLING SEQUENCE:
;       EBF_READ, filename, label, data [, ALIASES=Aliases] [, /APPEND] [,  /REORDER] [, STATUS=Status]
;
; PARAMS:
;    filename- A string specifying the filename to read
;    label-    A string specifying the path (if ending with "/") or the name of
;              the data object to be read from the ebf file.
;
; RETURN VALUE:
;       data: An array if tagname does not end with '/' or else a
;       structure containg the data objects in specified path
;
;       STATUS :  In event of I/O failure this is 0 else 1
;
;
; KEYWORDS:
;       ALIASES : This makes it possible to rename the tags in the loaded
;                 structure for example naming 'feh' as 'metals'. The
;                 number of tagnames (labels) should be equal to number of
;                 aliases. When multiple labels are read this must be set.
;       APPEND :  Will append items to an existing structure
;                 insteading of creating a new variable. Structure
;                 should not be named as XXX
;
;       REORDER : Will transpose the loaded data if it is a
;                 multidimenisonal array of fundamental data
;                 types (int,float etc). Structure arrays or fields
;                 of structures cannot be transposed.
;
;       DATAUNIT: If units exist then they will be returned in this
;                 keyword or else a blank string.
;-

ebf_write

;+
;
; NAME:
;       EBF_WRITE
;
; PURPOSE:
;          Writes a data specified by a tag name to a .ebf files.
;
; DESCRIPTION:
;       Data objects in an ebf file are stored like unix filenames
;       starting with / as root directory e.g.  '/FeH',  '/SDSS/g' ,
;       '/SDSS/r'. Using this analogy files correspond to data objects.
;
; CALLING SEQUENCE:
;       ebf_write,filename,label,data,[TAGNAMES=,ALIASES=,APPEND=,REORDER=,STATUS=,DATAUNIT=]
;
; EXAMPLES:
;         see examples in ebf_read.pro
;
; KEYWORDS:
;
;       tagname: If one only wants to write specific items
;                 then these can be specified as string array
;
;       aliases: This makes it possible to assign labels in the
;                 ebf file different from structure filed names
;                 structure for example naming 'mystruct.feh'
;                  as '"/metals'. The number of aliases should be
;                  equal to number of items being written.
;
;       append :  To append an item to an existing file
;
;       reorder : To transpose and write a multidimensional array. A
;                 structure array or its elements cannot be transposed.
;
;       dataunit: a string or an array of string specifying units.
;                 The number of units should be equal to number of
;                 items being written
;
;       status :  In event of I/O failure this is 0 else 1
;
; PARAMS:
;
;       filename : A string specifying the filename to write to.
;
;       tagname :  A string specifying the tagname of the data to be
;       written
;
;       data :     Data to be stored. Single or mutidimensional arrays of long,
;                 long64, float, double and byte types. Or
;                 structures. Recursive structures are also allowed.
;

ebf_info

;+
; NAME:
;       EBF_INFO
;
; PURPOSE:
;       This procedure prints a summary of the  different data objects
;       in the .ebf file.
;
; CALLING SEQUENCE:
;       ebf_info,filename
;
; INPUTS:
;       filename: A string specifying the filename to read
;

ebf_containskey

;+
; NAME:
;       EBF_containskey
;
; PURPOSE:
;       This function checks if a data item is present in a file
;
; CALLING SEQUENCE:
;       result=ebf_containskeys(filename,dataname)
;
; INPUTS:
;    filename- A string specifying the filename to read
;
; OUTPUT:
;       an array of strings
;
;
; EXAMPLE:
;       IDL> result=ebf_containskey("test.ebf","/x")
;