Programming in Go

Contents

Programming YottaDB in the Go language is accomplished through a wrapper for Simple API threaded functions that uses cgo to provide a “yottadb” package for access from Go application code. The wrapper must be installed on a system after YottaDB is installed.

There are two Go APIs:

  • Go Easy API aims to be a straightforward, easy-to-use API to access YottaDB without limiting the functionality of YottaDB. The Go Easy API consists of Go Easy API Functions that use standard Go data types and structures.
  • Go Simple API aims to improve performance by reducing copying between Go and YottaDB heaps by defining structures BufferT, BufferTArray, and KeyT which contain pointers to structures and data in the YottaDB heap. Go Simple API functionality is provided by Go methods where a method can meaningfully be associated with a structure, and by Go functions where there is no meaningful association with a structure.

Except for triggers, which are written in M and which can exist in the same process as Go code because they run in a special, isolated, environment, Go and M code in the same processs is not supported.

As the Go language has important differences from C (for example, it has structures with methods but lacks macros), below are Go-specific sections of the Quick Start, Concepts, Symbolic Constants, Data Structures & Type Definitions, Simple API and Utility Functions sections. The sections that are specific to Go are intended to supplement, but not subsume, their C counterparts.

Go application code must not directly use the YottaDB C API structures and functions (those prefixed by C. or described in the C Simple API above) as such usage bypasses important controls, but should instead use the structures, methods and functions exposed by the YottaDB Go wrapper. C. prefixed structures and functions are mentioned only for clarity in documentation and brevity of explanation. For example, C.ydb_buffer_t is the C ydb_buffer_t structure defined in Data Structures & Type Definitions.

All subsections of the Programming in Go section are prefixed with “Go” to ensure unique names for hyperlinking.

As Go implementations are inherently multi-threaded, where the C Simple API provides separate functions for use in multi-threaded applications, e.g., ydb_get_s() vs. ydb_get_st(), the Go wrapper wraps the function for use in multi-threaded applications. Also, as Go is multi-threaded, calls include an errstr parameter to get the correct $zstatus for each call.

Go Quick Start

The YottaDB Go wrapper requires a minimum YottaDB version of r1.24 and is tested with a minimum Go version of 1.11. If the Golang packages on your operating system are older, and the Go wrapper does not work, please obtain and install a newer Golang implementation.

The Go Quick Start assumes that YottaDB has already been installed as described in the Quick Start section. After completing step 1 (Installing YottaDB), download the Go wrapper, install it and test it.

$ go get lang.yottadb.com/go/yottadb
$ go build lang.yottadb.com/go/yottadb
$ source $(pkg-config --variable=prefix yottadb)/ydb_env_set
$ go get -t lang.yottadb.com/go/yottadb
$ go test lang.yottadb.com/go/yottadb
ok      lang.yottadb.com/go/yottadb     0.194s
$

There are a number of programs in the go/src/lang.yottadb.com/go/yottadb directory that you can look at.

  1. Put your GO program in a directory of your choice, e.g., $ydb_dir directory and change to that directory. As a sample program, you can download the wordfreq.go program, with a reference input file and corresponding reference output file. Compile it thus: go build wordfreq.go.
  2. Run your program and verify that the output matches the reference output. For example:
$ cd $ydb_dir
$ wget https://gitlab.com/YottaDB/DB/YDBTest/raw/master/go/inref/wordfreq.go
$ go build wordfreq.go
$ wget https://gitlab.com/YottaDB/DB/YDBTest/raw/master/simpleapi/outref/wordfreq_input.txt
$ wget https://gitlab.com/YottaDB/DB/YDBTest/raw/master/simpleapi/outref/wordfreq_output.txt
$ ./wordfreq <wordfreq_input.txt >wordfreq_output_go.txt
$ diff wordfreq_output_go.txt wordfreq_output.txt
$

Note that the wordfreq.go program randomly uses local or global variables (see Local and Global Variables).

Go Concepts

As the YottaDB wrapper is distributed as a Go package, function calls to YottaDB are prefixed in Go code with yottadb. (e.g., application code to call the ValST() function would be written yottadb.ValST(…).

Go Error Interface

YottaDB has a comprehensive set of error return codes. Each has a unique number and a mnemonic. Thus, for example, to return an error that a buffer allocated for a return value is not large enough, YottaDB uses the INVSTRLEN error code, which has the numeric value yottadb.YDB_ERR_INVSTRLEN. YottaDB attempts to maintain stability of the numeric values and mnemonics from release to release, to ensure applications remain compatible when the underlying YottaDB releases are upgraded. While the Go error interface provides for a call to return an error as a string (with nil for a successful return), applications in other languages, such as C, expect a numeric return value.

Where C application code calling YottaDB functions will check the return code, and if it is not YDB_OK access the intrinsic special variable $zstatus for more detailed information (through the errstr parameter in a multi-threaded application), Go application code calling YottaDB methods and functions will check the error interface to determine whether it is nil. This means that Go application code will never see a yottadb.YDB_OK return.

The YottaDB Go error interface has a structure and a method. Sample usage:

 _, err := yottadb.ValE(yottadb.NOTTP, nil, "^hello", []string{})
if err != nil {
    errcode := yottadb.ErrorCode(err)
 }

In the documentation:

  • Error codes specific to each function are noted. However, common errors can also be returned. For example, while the Go BufferT ValStr() method can return INVSTRLEN, it can also return errors from the YottaDB engine.
  • An error name such as INVSTRLEN refers to the underlying error, whether application code references the numeric value or the string.

Go Symbolic Constants

YottaDB symbolic constants are available in the YottaDB package, for example, yottadb.YDB_ERR_INVSTRLEN.

NOTTP (yottadb.NOTTP) as a value for parameter tptoken indicates to the invoked YottaDB method or function that the caller is not inside a transaction.

Go Easy API

A global or local variable node, or an intrinsic special variable, is specified using the construct varname string, subary []string. For an intrinsic special variable, subary must be the null array, []string{}, or nil. For a global or local variable, a null array or nil for subary refers to the root node, the entire tree, or both, depending on the function and context.

As the Go Easy API involves more copying of data between the Go and YottaDB runtime systems, it requires the CPU to perform a little more work than the Go Simple API does. Whether or not this has a measurable impact on performance depends on the application and workload.

The length of strings (values and subscripts) in YottaDB is variable, as is the number of subscripts a local or global variable can have. The Go Simple API requires application code to allocate memory for buffers, raising errors when allocated memory (either size or number of buffers) is insufficient. Requiring application code using the Go Easy API to similarly allocate memory would be at odds with our goal of having it “just work”. Although YottaDB provides functionality to a priori determine the length of a value in order to allocate required memory, doing this for every call would adversely affect performance. The Go Easy API therefore allocates buffers initially of a size and number we believe to be reasonable. Whenever a result exceeds its allocation and returns an error, YottaDB expands the allocation transparently to the caller, and repeats the operation, remembering the expanded size for future allocations in the process.

Go Easy API Functions

Go DataE()

func DataE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) (uint32, error)

Matching Go DataST(), DataE() function wraps and returns the result of ydb_data_st() (0, 1, 10, or 11). In the event of an error, the return value is unspecified.

Go DeleteE()

func DeleteE(tptoken uint64,  errstr *BufferT,
        deltype int, varname string, subary []string) error

Matching Go DeleteST(), DeleteE() wraps ydb_delete_st() to delete a local or global variable node or (sub)tree, with a value of yottadb.YDB_DEL_NODE for deltype specifying that only the node should be deleted, leaving the (sub)tree untouched, and a value of yottadb.YDB_DEL_TREE specifying that the node as well as the (sub)tree are to be deleted.

Go DeleteExclE()

func DeleteExclE(tptoken uint64,
         errstr *BufferT, varnames []string) error

Matching Go DeleteExclST(), DeleteExclE() wraps ydb_delete_excl_st() to delete all local variables except those specified. In the event varnames has no elements (i.e., []string{}), DeleteExclE() deletes all local variables.

In the event that the number of variable names in varnames exceeds yottadb.YDB_MAX_NAMES, the error return is ERRNAMECOUNT2HI. Otherwise, if ydb_delete_excl_st() returns an error, the function returns the error.

As mixing M and Go application code in the same process is not supported, the warning in ydb_delete_excl_s() does not apply.

Go IncrE()

func IncrE(tptoken uint64, errstr *BufferT,
        incr, varname string, subary []string) (string, error)

Matching Go IncrST(), IncrE() wraps ydb_incr_st() to atomically increment the referenced global or local variable node coerced to a number with incr coerced to a number, with the result stored in the node and returned by the function.

  • If ydb_incr_st() returns an error such as NUMOFLOW or INVSTRLEN, the function returns the error.
  • Otherwise, it returns the incremented value of the node.

With a nil value for incr, the default increment is 1. Note that the value of the empty string coerced to an integer is zero, but 1 is a more useful default value for an omitted parameter in this case.

Go LockDecrE()

func LockDecrE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) error

Matching Go LockDecrST() LockDecrE() wraps ydb_lock_decr_st() to decrement the count of the lock name referenced, releasing it if the count goes to zero or ignoring the invocation if the process does not hold the lock.

Go LockE()

func LockE(tptoken uint64, errstr *BufferT,
        timeoutNsec uint64, namesnsubs ... interface{}) error

Matching Go LockST(), LockE() releases all lock resources currently held and then attempts to acquire the named lock resources referenced. If no lock resources are specified, it simply releases all lock resources currently held and returns.

interface{} is a series of pairs of varname string and subary []string parameters, where a null subary parameter ([]string{}) specifies the unsubscripted lock resource name.

If lock resources are specified, upon return, the process will have acquired all of the named lock resources or none of the named lock resources.

  • If timeoutNsec exceeds yottadb.YDB_MAX_TIME_NSEC, the function returns with an error return of TIME2LONG.
  • If the lock resource names exceeds the maximum number supported (currently eleven), the function returns a PARMOFLOW error.
  • If namesnsubs is not a series of alternating string and []string parameters, the function returns the INVLNPAIRLIST error.
  • If it is able to aquire the lock resource(s) within timeoutNsec nanoseconds, the function returns holding the lock resource(s); otherwise it returns LOCKTIMEOUT. If timeoutNsec is zero, the function makes exactly one attempt to acquire the lock resource(s).

Go LockIncrE()

func LockIncrE(tptoken uint64, errstr *BufferT,
        timeoutNsec uint64,
        varname string, subary []string) error

Matching Go LockIncrST(), LockIncrE() wraps ydb_lock_incr_st() to attempt to acquire the referenced lock resource name without releasing any locks the process already holds.

  • If the process already holds the named lock resource, the function increments its count and returns.
  • If timeoutNsec exceeds yottadb.YDB_MAX_TIME_NSEC, the function returns with an error return TIME2LONG.
  • If it is able to aquire the lock resource within timeoutNsec nanoseconds, it returns holding the lock, otherwise it returns LOCKTIMEOUT. If timeoutNsec is zero, the function makes exactly one attempt to acquire the lock.

Go NodeNextE()

func NodeNextE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) ([]string, error)

Matching Go NodeNextST(), NodeNextE() wraps ydb_node_next_st() to facilitate depth first traversal of a local or global variable tree. A node or subtree does not have to exist at the specified key.

  • If there is a next node, it returns the subscripts of that next node.
  • If there is no node following the specified node, the function returns the NODEEND error.

Go NodePrevE()

func NodePrevE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) ([]string, error)

Matching Go NodePrevST(), NodePrevE() wraps ydb_node_previous_st() to facilitate reverse depth first traversal of a local or global variable tree. A node or subtree does not have to exist at the specified key.

  • If there is a previous node, it returns the subscripts of that previous node; an empty string array if that previous node is the root.
  • If there is no node preceding the specified node, the function returns the NODEEND error.

Go SetValE()

func SetValE(tptoken uint64, errstr *BufferT,
        value, varname string, subary []string) error

Matching Go SetValST(), at the referenced local or global variable node, or the intrinsic special variable, SetValE() wraps ydb_set_st() to set the value specified by value.

Go SubNextE()

func SubNextE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) (string, error)

Matching Go SubNextST(), SubNextE() wraps ydb_subscript_next_st() to facilitate breadth-first traversal of a local or global variable sub-tree. A node or subtree does not have to exist at the specified key.

  • At the level of the last subscript, if there is a next subscript with a node and/or a subtree, it returns that subscript.
  • If there is no next node or subtree at that level of the subtree, the function returns the NODEEND error.

In the special case where subary is the null array, SubNextE() returns the name of the next global or local variable, and the NODEEND error if there is no global or local variable following varname.

Go SubPrevE()

func SubPrevE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) (string, error)

Matching Go SubPrevST(), SubPrevE() wraps ydb_subscript_previous_st() to facilitate reverse breadth-first traversal of a local or global variable sub-tree. A node or subtree does not have to exist at the specified key.

  • At the level of the last subscript, if there is a previous subscript with a node and/or a subtree, it returns that subscript.
  • If there is no previous node or subtree at that level of the subtree, the function returns the NODEEND error.

In the special case where subary is the null array SubNextE() returns the name of the previous global or local variable, and the NODEEND error if there is no global or local variable preceding varname.

Go TpE()

func TpE(tptoken uint64, errstr *BufferT,
        tpfn func(uint64, *BufferT) int32, transid string,
        varnames []string) error

Matching Go TpST(), TpE() wraps ydb_tp_st() to implement Transaction Processing.

Refer to Go TpST() for a more detailed discussion of YottaDB Go transaction processing.

Go ValE()

func ValE(tptoken uint64, errstr *BufferT,
        varname string, subary []string) (string, error)

Matching Go ValST(), ValE() wraps ydb_get_st() to return the value at the referenced global or local variable node, or intrinsic special variable.

  • If ydb_get_s() returns an error such as GVUNDEF, INVSVN, LVUNDEF, the function returns the error.
  • Otherwise, it returns the value at the node.

Go Simple API

The Go Simple API consists of Go Data Structures & Type Definitions, Go Simple API Access Methods, Go Simple API BufferT Methods, Go Simple API BufferTArray Methods, Go Simple API KeyT Methods and Go Simple API Functions. Each of them wraps a function in the C Simple API – refer to the descriptions of those functions for more detailed information. The majority of the functionality is in Go Simple API KeyT Methods.

Go Data Structures & Type Definitions

The C.ydb_buffer_t structure, which is the ydb_buffer_t structure described in Data Structures & Type Definitions is used to pass values between Go application code and YottaDB. The design pattern is that the ydb_buffer_t structures are in memory managed by YottaDB. Go structures contain pointers to the YottaDB structures so that when the Go garbage collector moves Go structures, the pointers they contain remain valid.

There are three structures for the interface between YottaDB and Go: BufferT for data, BufferTArray for a list of subscripts or a set of variable names, KeyT for keys where a key in turn consists of a variable or lock resource name and subscripts, as discussed in Concepts.

Methods for each structure are classified as either Go Simple API Access Methods or Go Simple API methods. Go Simple API Access Methods are methods implemented in the Go wrapper for managing the structures for data interchange. Go Simple API methods wrap functionality exposed by the YottaDB API.

Go Simple API Access Methods

Go Simple API Access Methods for BufferT

Go BufferT Alloc()
func (buffer *BufferT) Alloc(nBytes uint32)

Allocate a buffer in YottaDB heap space of size nBytes; and set BufferT structure to provide access to that buffer.

Go BufferT Dump()
func (buffer *BufferT) Dump()

For debugging purposes, dump on stdout:

  • cbuft as a hexadecimal address;
  • for the C.ydb_buffer_t structure referenced by cbuft:
    • buf_addr as a hexadecimal address, and
    • len_alloc and len_used as integers; and
  • at the address buf_addr, the lower of len_used or len_alloc bytes in zwrite format.

As this function is intended for debugging and provides details of internal structures, its output is likely to change as internal implementations change, even when stability of the external API is maintained.

Go BufferT DumpToWriter()
func (buffer *BufferT) DumpToWriter(writer io.writer)

For debugging purposes, dump on writer:

  • cbuft as a hexadecimal address;
  • for the C.ydb_buffer_t structure referenced by cbuft:
    • buf_addr as a hexadecimal address, and
    • len_alloc and len_used as integers; and
  • at the address buf_addr, the lower of len_used or len_alloc bytes in zwrite format.

As this function is intended for debugging and provides details of internal structures, its output is likely to change as internal implementations change, even when stability of the external API is maintained.

Go BufferT Free()
func (buffer *BufferT) Free()

The inverse of the Alloc() method: release the buffer in YottaDB heap space referenced by the C.ydb_buffer_t structure, release the C.ydb_buffer_t, and set cbuft in the BufferT structure to nil.

Go BufferT BufferTFromPtr()
func (buffer *BufferT) BufferTFromPtr(errstr unsafe.Pointer)

This method is intended for use in advanced cases, such as those encountered internally to the wrapper, when a BufferT object must wrap an existing C.ydb_buffer_t struct.

Note: Modifying errstr, or accessing memory it references may lead to code that behaves unpredictably and is hard to debug.

Go BufferT LenAlloc()
func (buffer *BufferT) LenAlloc(tptoken uint64,
        errstr *BufferT) (uint32, error)
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • Otherwise, return the len_alloc field of the C.ydb_buffer_t structure referenced by cbuft.
Go BufferT LenUsed()
func (buffer *BufferT) LenUsed(tptoken uint64,
        errstr *BufferT) (uint32, error)
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If the len_used field of the C.ydb_buffer_t structure is greater than its len_alloc field (owing to a prior INVSTRLEN error), return an INVSTRLEN error and the len_used field of the C.ydb_buffer_t structure referenced by cbuft.
  • Otherwise, return the len_used field of the C.ydb_buffer_t structure referenced by cbuft.
Go BufferT SetLenUsed()
func (buffer *BufferT) SetLenUsed(tptoken uint64,
        errstr *BufferT, newLen uint32) error

Use this method to change the length of a used substring of the contents of the buffer referenced by the buf_addr field of the referenced C.ydb_buffer_t.

  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If newLen is greater than the len_alloc field of the referenced C.ydb_buffer_t, make no changes and return with an error return of INVSTRLEN.
  • Otherwise, set the len_used field of the referenced C.ydb_buffer_t to newLen.

Note that even if newLen is not greater than the value of len_alloc, setting a len_used value greater than the number of meaningful bytes in the buffer will likely lead to hard-to-debug errors.

Go BufferT SetValBAry()
func (buffer *BufferT) SetValBAry(tptoken uint64,
        errstr *BufferT, val *[]byte) error
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If the length of val is greater than the len_alloc field of the C.ydb_buffer_t structure referenced by cbuft, make no changes and return INVSTRLEN.
  • Otherwise, copy the bytes of val to the referenced buffer and set the len_used field to the length of val.
Go BufferT SetValStr()
func (buffer *BufferT) SetValStr(tptoken uint64,
        errstr *BufferT, val *string) error
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If the length of val is greater than the len_alloc field of the C.ydb_buffer_t structure referenced by cbuft, make no changes and return INVSTRLEN.
  • Otherwise, copy the bytes of val to the referenced buffer and set the len_used field to the length of val.
Go BufferT SetValStrLit()
func (buffer *BufferT) SetValStrLit(tptoken uint64,
        errstr *BufferT, val string) error
  • If the the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If the length of val is greater than the len_alloc field of the C.ydb_buffer_t structure referenced by cbuft, make no changes and return INVSTRLEN.
  • Otherwise, copy the bytes of val to the referenced buffer and set the len_used field to the length of val.
Go BufferT ValBAry()
func (buffer *BufferT) ValBAry(tptoken uint64,
        errstr *BufferT) (*[]byte, error)
  • If the the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If the len_used field of the C.ydb_buffer_t structure is greater than its len_alloc field (owing to a prior INVSTRLEN error), return an INVSTRLEN error and len_alloc bytes as a byte array.
  • Otherwise, return len_used bytes of the buffer as a byte array.
Go BufferT ValStr()
func (buffer *BufferT) ValStr(tptoken uint64,
        errstr *BufferT) (*string, error)
  • If the the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If the len_used field of the C.ydb_buffer_t structure is greater than its len_alloc field (owing to a prior INVSTRLEN error), return an INVSTRLEN error and len_alloc bytes as a string.
  • Otherwise, return len_used bytes of the buffer as a string.

Go Simple API Access Methods for BufferTArray

Go BufferTArray Alloc()
func (buftary *BufferTArray) Alloc(numBufs, nBytes uint32)

Allocate an array of numSubs buffers in YottaDB heap space, each of of size bufSiz, referenced by the BufferTArray structure.

Go BufferTArray Dump()
func (buftary *BufferTArray) Dump()

For debugging purposes, dump on stdout:

  • cbuftary as a hexadecimal address;
  • elemsAlloc and elemsUsed as integers;
  • for each element of the smaller of elemsAlloc and elemsUsed elements of the C.ydb_buffer_t array referenced by cbuftary:
    • buf_addr as a hexadecimal address, and
    • len_alloc and len_used as integers; and
    • the smaller of len_used and len_alloc bytes at the address buf_addr, in zwrite format.

As this function is intended for debugging and provides details of internal structures, its output is likely to change as internal implementations change, even when stability of the external API is maintained.

Go BufferTArray DumpToWriter()
func (buftary *BufferTArray) DumpToWriter(writer io.writer)

For debugging purposes, dump on writer:

  • cbuftary as a hexadecimal address;
  • elemsAlloc and elemsUsed as integers;
  • for each element of the smaller of elemsAlloc and elemsUsed elements of the C.ydb_buffer_t array referenced by cbuftary:
    • buf_addr as a hexadecimal address, and
    • len_alloc and len_used as integers; and
    • the smaller of len_used and len_alloc bytes at the address buf_addr, in zwrite format.

As this function is intended for debugging and provides details of internal structures, its output is likely to change as internal implementations change, even when stability of the external API is maintained.

Go BufferTArray ElemAlloc()
func (buftary *BufferTArray) ElemAlloc() uint32
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • Otherwise, return the number of allocated buffers.
Go BufferTArray ElemLenAlloc()
func (buftary *BufferTArray)
        ElemLenAlloc(tptoken uint64) uint32
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • Otherwise, return the len_alloc from the C.ydb_buffer_t structures referenced by cbuftary, all of which have the same value.
Go BufferTArray ElemLenUsed()
func (buftary *BufferTArray) ElemLenUsed(tptoken uint64,
        errstr *BufferT, idx uint32) (uint32, error)
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to the elemsAlloc of the BufferTArray structure, return with an error return of INSUFFSUBS.
  • Otherwise, return the len_used field of the array element specifed by idx of the C.ydb_buffer_t array referenced by cbuftary.
Go BufferTArray ElemUsed()
func (buftary *BufferTArray) ElemUsed() uint32
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • Otherwise, return the value of the elemsUsed field.
Go BufferTArray Free()
func (buftary *BufferTArray) Free()

The inverse of the Alloc() method: release the numSubs buffers and the C.ydb_buffer_t array. Set cbuftary to nil, and elemsAlloc and elemsUsed to zero.

Go BufferTArray SetElemLenUsed()
func (buftary *BufferTArray)
        SetElemLenUsed(tptoken uint64, errstr *BufferT,
        idx, newLen uint32) error

Use this method to set the number of bytes in C.ydb_buffer_t structure referenced by cbuft of the array element specified by idx, for example to change the length of a used substring of the contents of the buffer referenced by the buf_addr field of the referenced C.ydb_buffer_t.

  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to elemsAlloc, make no changes and return an INSUFFSUBS error.
  • If newLen is greater than the len_alloc field of the referenced C.ydb_buffer_t, make no changes and return an INVSTRLEN error.
  • Otherwise, set the len_used field of the referenced C.ydb_buffer_t to newLen.

Note that even if newLen is not greater than the value of len_alloc, using a len_used value greater than the number of meaningful bytes in the buffer will likely lead to hard-to-debug errors.

Go BufferTArray SetElemUsed()
func (buftary *BufferTArray)
        SetElemUsed(tptoken uint64, errstr *BufferT,
        newUsed uint32) error

Use this method to set the current number of valid strings (subscripts or variable names) in the BufferTArray.

  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If newUsed is greater than elemsAlloc, make no changes and return with an error return of INSUFFSUBS.
  • Otherwise, set elemsUsed to newUsed.

Note that even if newUsed is not greater than the value of elemsAlloc, using an elemsUsed value greater than the number of valid values in the array will likely lead to hard-to-debug errors.

Go BufferTArray SetValBAry()
func (buftary *BufferTArray) SetValBAry(tptoken uint64,
        errstr *BufferT, idx uint32, val *[]byte) error
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to elemsAlloc make no changes and return with an error return of INSUFFSUBS.
  • If the length of val is greater than the len_alloc field of the array element specified by idx, make no changes, and return INVSTRLEN.
  • Otherwise, copy the bytes of val to the referenced buffer and set the len_used field to the length of val.
Go BufferTArray SetValStr()
func (buftary *BufferTArray)
        SetValStr(tptoken uint64, errstr *BufferT,
        idx uint32, value *string) error
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to elemsAlloc make no changes and return with an error return of INSUFFSUBS.
  • If the length of val is greater than the len_alloc field of the array element specified by idx, make no changes, and return INVSTRLEN.
  • Otherwise, copy the bytes of val to the referenced buffer and set the len_used field to the length of val.
Go BufferTArray SetValStrLit()
func (buftary *BufferTArray)
        SetValStrLit(tptoken uint64, errstr *BufferT,
        idx uint32, value string) error
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to elemsAlloc make no changes and return with an error return of INSUFFSUBS.
  • If the length of val is greater than the len_alloc field of the C.ydb_buffer_t structure indexed by idx and referenced by cbuft, make no changes and return INVSTRLEN.
  • Otherwise, copy the bytes of val to the referenced buffer and set the len_used field to the length of val.
Go BufferTArray ValBAry()
func (buftary *BufferTArray)
        ValBAry(tptoken uint64, errstr *BufferT,
        idx uint32) (*[]byte, error)
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to elemsAlloc, return a zero length byte array and an error return of INSUFFSUBS.
  • If the len_used field of the C.ydb_buffer_t structure specified by idx is greater than its len_alloc field (owing to a previous INVSTRLEN error), return a byte array containing the len_alloc bytes at buf_addr and an INVSTRLEN error.
  • Otherwise, return a byte array containing the len_used bytes at buf_addr.
Go BufferTArray ValStr()
func (buftary *BufferTArray)
        ValStr(tptoken uint64, errstr *BufferT,
        idx uint32) (*string, error)
  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If idx is greater than or equal to elemsAlloc, return a zero length string and an error return of INSUFFSUBS.
  • If the len_used field of the C.ydb_buffer_t structure specified by idx is greater than its len_alloc field (owing to a previous INVSTRLEN error), return a string containing the len_alloc bytes at buf_addr and the INVSTRLEN error.
  • Otherwise, return a string containing the len_used bytes at buf_addr.

Go Simple API Access Methods for KeyT

As the members of KeyT are visible to Go programs (they start with upper-case letters), and application code can call the BufferT methods on Varnm and BufferTArray methods on SubAry, the Go KeyT Alloc(), Go KeyT Dump() and Go KeyT Free() methods are available for programming convenience.

Go KeyT Alloc()
func (key *KeyT) Alloc(varSiz, numSubs, subSiz uint32)

Invoke Varnm.Alloc(varSiz) (see Go BufferT Alloc()) and SubAry.Alloc(numSubs, subSiz) (see Go BufferTArray Alloc()).

Go KeyT Dump()
func (key *KeyT) Dump()

Invoke Varnm.Dump() (see Go BufferT Dump()) and SubAry.Dump() (see Go BufferTArray Dump()).

Go KeyT DumpToWriter()
func (key *KeyT) DumpToWriter(writer io.writer)

Invoke Varnm.Dump() (see Go BufferT Dump()) and SubAry.Dump() (see Go BufferTArray Dump()), sending the output to writer.

Go KeyT Free()
func (key *KeyT) Free()

Invoke Varnm.Free() (see Go BufferT Free()) and SubAry.Free() (see Go BufferTArray Free()).

Go Simple API BufferT Methods

Go Str2ZwrST()

func (buft *BufferT) Str2ZwrST(tptoken uint64,
        errstr *BufferT, zwr *BufferT) error

The method wraps ydb_str2zwr_st() to provide the string in zwrite format.

  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If len_alloc is not large enough, set len_used to the required length, and return an INVSTRLEN error. In this case, len_used will be greater than len_alloc until corrected by application code, and the value referenced by zwr is unspecified.
  • Otherwise, set the buffer referenced by buf_addr to the zwrite format string, and set len_used to the length.

Note that the length of a string in zwrite format is always greater than or equal to the string in its original, unencoded format.

Go Zwr2StrST()

func (buft *BufferT) Zwr2StrST(tptoken uint64,
        errstr *BufferT, str *BufferT) error

This method wraps ydb_zwr2str_st() and is the inverse of Go Str2ZwrST().

  • If the underlying structures have not yet been allocated, return the STRUCTNOTALLOCD error.
  • If len_alloc is not large enough, set len_used to the required length, and return an INVSTRLEN error. In this case, len_used will be greater than len_alloc until corrected by application code.
  • If str has errors and is not in valid zwrite format, set len_used to zero, and return the error code returned by ydb_zwr2str_s() e.g., INVZWRITECHAR`.
  • Otherwise, set the buffer referenced by buf_addr to the unencoded string, set len_used to the length.

Go Simple API BufferTArray Methods

Go DeleteExclST()

func (buftary *BufferTArray)
        DeleteExclST(tptoken uint64, errstr *BufferT) error

DeleteExclST() wraps ydb_delete_excl_st() to delete all local variable trees except those of local variables whose names are specified in the BufferTArray structure. In the special case where elemsUsed is zero, the method deletes all local variable trees.

In the event that the elemsUsed exceeds yottadb.YDB_MAX_NAMES, the error return is ERRNAMECOUNT2HI.

As mixing M and Go application code in the same process is not supported, the warning in ydb_delete_excl_s() does not apply.

Go TpST()

func (buftary *BufferTArray) TpST(tptoken uint64,
        errstr *BufferT, tpfn func(uint64, *BufferT) int,
        transid string) error

TpST() wraps ydb_tp_st() to implement Transaction Processing. tpfn is a function with two parameters, the first of which is a tptoken and the second is an errstr.

As an alternative to parameters for tpfn, create closures.

A function implementing logic for a transaction should return int with one of the following:

  • A normal return (YDB_OK) to indicate that per application logic, the transaction can be committed. The YottaDB database engine will commit the transaction if it is able to, as discussed in Transaction Processing, and if not, will call the function again.
  • YDB_TP_RESTART to indicate that the transaction should restart, either because application logic has so determined or because a YottaDB function called by the function has returned TPRESTART.
  • YDB_TP_ROLLBACK to indicate that TpST() should not commit the transaction, and should return ROLLBACK to the caller.

The BufferTArray receiving the TpST() method is a list of local variables whose values should be saved, and restored to their original values when the transaction restarts. If the cbuftary structures have not been allocated or elemsUsed is zero, no local variables are saved and restored; and if elemsUsed is 1, and that sole element references the string “*” all local variables are saved and restored.

A case-insensitive value of “BA” or “BATCH” for transid indicates to YottaDB that it need not ensure Durability for this transaction (it continues to ensure Atomicity, Consistency, and Isolation), as discussed under ydb_tp_st().

Please see both the description of ydb_tp_st() and the sections on Transaction Processing and Threads and Transaction Processing for details.

Note

If the transaction logic receives a YDB_TP_RESTART or YDB_TP_ROLLBACK from a YottaDB function or method that it calls, it must return that value to the calling TpE() or TpST(). Failure to do so could result in application level data inconsistencies and hard to debug application code.

Go Simple API KeyT Methods

KeyT methods return errors returned by methods that invoke the underlying Varnm and SubAry members of KeyT structures.

Go DataST()

func (key *KeyT) DataST(tptoken uint64,
        errstr *BufferT) (uint32, error)

Matching Go DataE(), DataST() returns the result of ydb_data_st() (0, 1, 10, or 11). In the event of an error, the return value is unspecified.

Go DeleteST()

func (key *KeyT) DeleteS(tptoken uint64,
        errstr *BufferT, deltype int) error

Matching Go DeleteE(), DeleteST() wraps ydb_delete_st() to delete a local or global variable node or (sub)tree, with a value of yottadb.YDB_DEL_NODE for deltype specifying that only the node should be deleted, leaving the (sub)tree untouched, and a value of yottadb.YDB_DEL_TREE specifying that the node as well as the (sub)tree are to be deleted.

Go IncrST()

func (key *KeyT) IncrST(tptoken uint64,
        errstr *BufferT, incr, retval *BufferT) error

Matching Go IncrE(), IncrST() wraps ydb_incr_st() to atomically increment the referenced global or local variable node coerced to a number, with incr coerced to a number. It stores the result in the node and also returns it through the BufferT structure referenced by retval.

  • If ydb_incr_st() returns an error such as NUMOFLOW, the method makes no changes to the structures under retval and returns the error.
  • If the length of the data to be returned exceeds retval.lenAlloc, the method sets the len_used of the C.ydb_buffer_t referenced by retval to the required length, and returns an INVSTRLEN error. The value referenced by retval is unspecified.
  • Otherwise, it copies the data to the buffer referenced by the retval.buf_addr, sets retval.lenUsed to its length.

With a nil value for incr, the default increment is 1. Note that the value of the empty string coerced to an integer is zero, but 1 is a more useful default value for an omitted parameter in this case.

Go LockDecrST()

func (key *KeyT) LockDecrS(tptoken uint64,
        errstr *BufferT) error

Matching Go LockDecrE() LockDecrST() wraps ydb_lock_decr_st() to decrement the count of the lock name referenced, releasing it if the count goes to zero or ignoring the invocation if the process does not hold the lock.

Go LockIncrST()

func (key *KeyT) LockIncrST(tptoken uint64,
        errstr *BufferT, timeoutNsec uint64) error

Matching Go LockIncrE(), LockIncrST() wraps ydb_lock_incr_st() to attempt to acquire the referenced lock resource name without releasing any locks the process already holds.

  • If the process already holds the named lock resource, the method increments its count and returns.
  • If timeoutNsec exceeds yottadb.YDB_MAX_TIME_NSEC, the method returns with an error return TIME2LONG.
  • If it is able to aquire the lock resource within timeoutNsec nanoseconds, it returns holding the lock, otherwise it returns LOCK_TIMEOUT. If timeoutNsec is zero, the method makes exactly one attempt to acquire the lock.

Go NodeNextST()

func (key *KeyT) NodeNextST(tptoken uint64,
        errstr *BufferT, next *BufferTArray) error

Matching Go NodeNextE(), NodeNextST() wraps ydb_node_next_st() to facilitate depth first traversal of a local or global variable tree. A node or subtree does not have to exist at the specified key.

  • If there is a subsequent node:
    • If the number of subscripts of that next node exceeds next.elemsAlloc, the method sets next.elemsUsed to the number of subscripts required, and returns an INSUFFSUBS error. In this case the elemsUsed is greater than elemsAlloc.
    • If one of the C.ydb_buffer_t structures referenced by next (call the first or only element n) has insufficient space for the corresponding subscript, the method sets next.elemsUsed to n, and the len_alloc of that C.ydb_buffer_t structure to the actual space required. The method returns an INVSTRLEN error. In this case the len_used of that structure is greater than its len_alloc.
    • Otherwise, it sets the structure next to reference the subscripts of that next node, and next.elemsUsed to the number of subscripts.
  • If there is no subsequent node, the method returns the NODEEND error (yottadb.YDB_ERR_NODEEND), making no changes to the structures below next.

Go NodePrevST()

func (key *KEyT) NodePrevST(tptoken uint64,
        errstr *BufferT, prev *BufferTArray) error

Matching Go NodePrevE(), NodePrevST() wraps ydb_node_previous_st() to facilitate reverse depth first traversal of a local or global variable tree. A node or subtree does not have to exist at the specified key.

  • If there is a previous node:
    • If the number of subscripts of that previous node exceeds prev.elemsAlloc, the method sets prev.elemsUsed to the number of subscripts required, and returns an INSUFFSUBS error. In this case the elemsUsed is greater than elemsAlloc.
    • If one of the C.ydb_buffer_t structures referenced by prev (call the first or only element n) has insufficient space for the corresponding subscript, the method sets prev.elemsUsed to n, and the len_alloc of that C.ydb_buffer_t structure to the actual space required. The method returns an INVSTRLEN error. In this case the len_used of that structure is greater than its len_alloc.
    • Otherwise, it sets the structure prev to reference the subscripts of that prev node, and prev.elemsUsed to the number of subscripts.
  • If there is no previous node, the method returns the NODEEND error making no changes to the structures below prev.

Go SetValST()

func (key *KeyT) SetST(tptoken uint64,
        errstr *BufferT, value *BufferT) error

Matching Go SetValE(), at the referenced local or global variable node, or the intrinsic special variable, SetValST() wraps ydb_set_st() to set the value specified by value.

Go SubNextST()

func (key *KeyT) SubNextST(tptoken uint64,
        errstr *BufferT, retval *BufferT) error

Matching Go SubNextE(), SubNextST() wraps ydb_subscript_next_st() to facilitate breadth-first traversal of a local or global variable sub-tree. A node or subtree does not have to exist at the specified key.

  • At the level of the last subscript, if there is a next subscript with a node and/or a subtree:
    • If the length of that next subscript exceeds sub.len_alloc, the method sets sub.len_used to the actual length of that subscript, and returns an INVSTRLEN error. In this case sub.len_used is greater than sub.len_alloc.
    • Otherwise, it copies that subscript to the buffer referenced by sub.buf_addr, and sets sub.len_used to its length.
  • If there is no next node or subtree at that level of the subtree, the method returns the NODEEND error.

Go SubPrevST()

func (key *KeyT) SubPrevST(tptoken uint64,
        errstr *BufferT, retval *BufferT) error

Matching Go SubPrevE(), SubPrevST() wraps ydb_subscript_previous_st() to facilitate reverse breadth-first traversal of a local or global variable sub-tree. A node or subtree does not have to exist at the specified key.

  • At the level of the last subscript, if there is a previous subscript with a node and/or a subtree:
    • If the length of that previous subscript exceeds sub.len_alloc, the method sets sub.len_used to the actual length of that subscript, and returns an INVSTRLEN error. In this case sub.len_used is greater than sub.len_alloc.
    • Otherwise, it copies that subscript to the buffer referenced by sub.buf_addr, and sets buf.len_used to its length.
  • If there is no previous node or subtree at that level of the subtree, the method returns the NODEEND error.

Go ValST()

func (key *KeyT) ValST(tptoken uint64,
        errstr *BufferT, retval *BufferT) error

Matching Go ValE(), ValST() wraps ydb_get_st() to return the value at the referenced global or local variable node, or intrinsic special variable, in the buffer referenced by the BufferT structure referenced by retval.

  • If ydb_get_st() returns an error such as GVUNDEF, INVSVN, LVUNDEF, the method makes no changes to the structures under retval and returns the error.
  • If the length of the data to be returned exceeds retval.GetLenAlloc(), the method sets the len_used of the C.ydb_buffer_t referenced by retval to the required length, and returns an INVSTRLEN error.
  • Otherwise, it copies the data to the buffer referenced by the retval.buf_addr, and sets retval.lenUsed to the length of the returned value.

Go Simple API Functions

Go LockST()

func LockST(tptoken uint64, errstr *BufferT,
        timeoutNsec uint64, lockname ... *KeyT) error

Matching Go LockE(), LockST() wraps ydb_lock_st() to release all lock resources currently held and then attempt to acquire the named lock resources referenced. If no lock resources are specified, it simply releases all lock resources currently held and returns.

If lock resources are specified, upon return, the process will have acquired all of the named lock resources or none of the named lock resources.

  • If timeoutNsec exceeds yottadb.YDB_MAX_TIME_NSEC, the method returns with a TIME2LONG error.
  • If the number of lock resource names exceeds the maximum number supported (currently eleven), the function returns a PARMOFLOW error.
  • If it is able to aquire the lock resource(s) within timeoutNsec nanoseconds, it returns holding the lock resource(s); otherwise it returns LOCKTIMEOUT. If timeoutNsec is zero, the method makes exactly one attempt to acquire the lock resource(s).

Go Comprehensive API

The Go Comprehensive API is a project for the future, to follow the C Comprehensive API

Go Utility Functions

Go Error()

func (err *YDBError) Error() string

Error() is a method to return the expected error message string.

Go ErrorCode()

func ErrorCode(err error) int

ErrorCode() is a function used to find the error return code.

Go Exit()

func Exit() error

For a process that wishes to close YottaDB databases and no longer use YottaDB, the function wraps ydb_exit(). If ydb_exit() does not send a return value of YDB_OK, Exit() panics.

Although in theory typical processes should not need to call Exit() because normal process termination should close databases cleanly, in practice, thread shutdown may not always ensure that databases are closed cleanly, especially since the C atexit() functionality does not reliably work in Go’s multi-threaded environment. Application code should invoke Exit() prior to process exit, or when an application intends to continue with other work beyond use of YottaDB, to ensure that databases are closed cleanly. To accomplish this, you should use a “defer yottadb.Exit()” statement early in the main routine’s initialization.

Go IsLittleEndian()

func IsLittleEndian() bool

The function returns true if the underlying computing infrastructure is little endian and false otherwise.

Go MessageT()

func Message(tptoken uint64, errstr *BufferT,
        status int) (string, error)

MessageT() returns the text template for the error number specified by status.

  • If status does not correspond to an error that YottaDB recognizes, it returns the error UNKNOWNSYSERR.
  • Otherwise, it returns the error message text template for the error number specified by status.

Go NewError()

func NewError(tptoken uint64, errstr *BufferT, errnum int) error

NewError() is a function to create a new YDBError from errstr and errnum, setting the two private fields in the returned YDBError to the provided values.

Go ReleaseT()

func ReleaseT(tptoken uint64, errstr *BufferT) string

Returns a string consisting of six space separated pieces to provide version information for the Go wrapper and underlying YottaDB release:

  • The first piece is always “gowr” to idenfify the Go wrapper.
  • The Go wrapper release number, which starts with “v” and is followed by three numbers separated by a period (“.”), e.g., “v0.90.0” mimicking Semantic Versioning. The first is a major release number, the second is a minor release number under the major release and the third is a patch level. Even minor and patch release numbers indicate formally released software. Odd minor release numbers indicate software builds from “in flight” code under development, between releases. Note that although they follow the same format, Go wrapper release numbers are different from the release numbers of the underlying YottaDB release as reported by $zyrelease.
  • The third through sixth pieces are $zyrelease from the underlying YottaDB release.

Go Programming Notes

These Go Programming Notes supplement rather than supplant the more general Programming Notes for C.

Goroutines

In order to avoid restricting Go applications to calling the single-threaded YottaDB engine from a single goroutine (which would be unnatural to a Go programmer), the YottaDB Go wrapper calls the functions of the C Simple API that support multi-threaded applications, and includes logic to maintain the integrity of the engine.

Directly calling YottaDB C API functions bypasses this protection, and may result in unpredictable results (Murphy says that unpredictable results will occur when you least expect them). Therefore, Go application code should only call the YottaDB API exposed in this Programming in Go section.

Goroutines in a process are dynamically mapped by the Go implementation to run on threads within that process. Since YottaDB resources are held by the process rather than by the thread or the Goroutine, refer to the Threads discussion about the need for applications to avoid race conditions when accessing YottaDB resources.