# Using the immudb SDK

# Connection and authentication

immudb runs on port 3323 as the default. The code samples below illustrate how to connect your client to the server and authenticate using default options and the default username and password. You can modify defaults on the immudb server in immudb.toml in the config folder.

# Mutual TLS

To enable mutual authentication, a certificate chain must be provided to both the server and client. That will cause each to authenticate with the other simultaneously. In order to generate certs, use the openssl tool: generate.sh (opens new window).

./generate.sh localhost mysecretpassword

This generates a list of folders containing certificates and private keys to set up a mTLS connection.

# Disable authentication

You also have the option to run immudb with authentication disabled. However, without authentication enabled, it's not possible to connect to a server already configured with databases and user permissions. If a valid token is present, authentication is enabled by default.

# State management

It's the responsibility of the immudb client to track the server state. That way it can check each verified read or write operation against a trusted state.

# Verify state signature

If immudb is launched with a private signing key, each signed request can be verified with the public key. In this way the identity of the server can be proven. Check state signature to see how to generate a valid key.

# Tamperproof reading and writing

You can read and write records securely using a built-in cryptographic verification.

# Verified get and set

The client implements the mathematical validations, while your application uses a traditional read or write function.

# Writing and reading

The format for writing and reading data is the same both in Set and VerifiedSet, just as it is for reading data both in both Get and VerifiedGet.

The only difference is that VerifiedSet returns proofs needed to mathematically verify that the data was not tampered. Note that generating that proof has a slight performance impact, so primitives are allowed without the proof. It is still possible get the proofs for a specific item at any time, so the decision about when or how frequently to do checks (with the Verify version of a method) is completely up to the user. It's possible also to use dedicated auditors to ensure the database consistency, but the pattern in which every client is also an auditor is the more interesting one.

# Get and set

# Get at and since a transaction

You can retrieve a key on a specific transaction with VerifiedGetAt and since a specific transaction with VerifiedGetSince.

# Transaction by index

It's possible to retrieve all the keys inside a specific transaction.

# Verified transaction by index

It's possible to retrieve all the keys inside a specific verified transaction.

# History

The fundamental property of immudb is that it's an append-only database. This means that an update is a new insert of the same key with a new value. It's possible to retrieve all the values for a particular key with the history command.

History accepts the following parameters:

  • Key: a key of an item
  • Offset: the starting index (excluded from the search). Optional
  • Limit: maximum returned items. Optional
  • Desc: items are returned in reverse order. Optional
  • SinceTx:

# Counting

Counting entries is not supported at the moment.

# Scan

The scan command is used to iterate over the collection of elements present in the currently selected database. Scan accepts the following parameters:

  • Prefix: prefix. If not provided all keys will be involved. Optional
  • SeekKey: initial key for the first entry in the iteration. Optional
  • Desc: DESC or ASC sorting order. Optional
  • Limit: maximum returned items. Optional
  • SinceTx: immudb will wait that the transaction provided by SinceTx be processed. Optional
  • NoWait: Default false. When true scan doesn't wait for the index to be fully generated and returns the last indexed value. Optional

To gain speed it's possible to specify noWait=true. The control will be returned to the caller immediately, without waiting for the indexing to complete. When noWait is used, keep in mind that the returned data may not be yet up to date with the inserted data, as the indexing might not have completed.

# References

SetReference is like a "tag" operation. It appends a reference on a key/value element. As a consequence, when we retrieve that reference with a Get or VerifiedGet the value retrieved will be the original value associated with the original key. Its VerifiedReference counterpart is the same except that it also produces the inclusion and consistency proofs.

# SetReference and VerifiedSetReference

# GetReference and VerifiedGetReference

When reference is resolved with get or verifiedGet in case of multiples equals references the last reference is returned.

# Resolving reference with transaction id

It's possible to bind a reference to a key on a specific transaction using SetReferenceAt and VerifiedSetReferenceAt

# Secondary indexes

On top of the key value store immudb provides secondary indexes to help developers to handle complex queries.

# Sorted sets

The sorted set data type provides simplest secondary index you can create with immudb. That's a data structure that represents a set of elements ordered by the score of each element, which is a floating point number. The score type is a float64 to accommodate the maximum number of uses cases. 64-bit floating point gives a lot of flexibility and dynamic range, at the expense of having only 53-bits of integer.

When an integer64 is cast to a float there could be a loss of precision, but the insertion order is guaranteed by the internal database index that is appended to the internal index key.

ZAdd can reference an item by key or by index.

ZScan accepts following arguments:

  • Set: the name of the collection
  • SeekKey: initial key for the first entry in the iteration. Optional
  • SeekScore: the min or max score for the first entry in the iteration, depending on Desc value. Optional
  • SeekAtTx: the tx id for the first entry in the iteration. Optional
  • InclusiveSeek: the element resulting from the combination of the SeekKey SeekScore and SeekAtTx is returned with the result. Optional
  • Desc: DESC or ASC sorting order. Optional
  • SinceTx: immudb will wait that the transaction provided by SinceTx be processed. Optional
  • NoWait: when true scan doesn't wait that txSinceTx is processed. Optional
  • MinScore: minimum score filter. Optional
  • MaxScore: maximum score filter. Optional
  • Limit: maximum number of returned items. Optional

Having the possibility to get data specifying a transaction id: AtTx, it’s the optimal way to retrieve the data, as it can be done with random access to it. And it can be made immediately after the transaction was committed or at any point in the future. When the transaction ID is unknown by the application and the query is made by key or key prefixes, it will be served through the index, depending on the insertion rate, it can be delayed or up to date with inserted data, using a big number in SinceTx with NoWait in true will mean that the query will be resolved by looking at the most recent indexed data, but if your query needs to be resolved after some transactions has been inserted, you can set SinceTx to specify up to which transaction the index has to be made for resolving it.

# Transactions

GetAll, SetAll and ExecAll are the foundation of transactions in immudb. They allow the execution of a group of commands in a single step, with two important guarantees:

  • All the commands in a transaction are serialized and executed sequentially. No request issued by another client can ever interrupt the execution of a transaction. This guarantees that the commands are executed as a single isolated operation.
  • Either all of the commands are processed, or none are, so the transaction is also atomic.

# GetAll

# SetAll

A more versatile atomic multi set operation

SetBatch and GetBatch example

# ExecAll

ExecAll permits many insertions at once. The difference is that is possible to specify a list of a mix of key value set, reference and zAdd insertions. The argument of a ExecAll is an array of the following types:

  • Op_Kv: ordinary key value item
  • Op_ZAdd: ZAdd option element
  • Op_Ref: Reference option element

It's possible to persist and reference items that are already persisted on disk. In that case is mandatory to provide the index of the referenced item. This has to be done for:

  • Op_ZAdd
  • Op_Ref If zAdd or reference is not yet persisted on disk it's possible to add it as a regular key value and the reference is done onFly. In that case if BoundRef is true the reference is bounded to the current transaction values.

# Tx Scan

TxScan permits iterating over transactions.

The argument of a TxScan is an array of the following types:

  • InitialTx: initial transaction id
  • Limit: number of transactions returned
  • Desc: order of returned transacations

# Tamperproofing utilities

# Current State

CurrentState returns the last state of the server.

# Streams

Immudb provides stream capabilities. Internally it uses “delimited” messages technique, every chunk has a trailer that describe the length of the message. In this way the receiver can recompose chunk by chunk the original payload. Stream methods accepts a readers as a part of input and output arguments. In this way the large value is decomposed in small chunks that are streamed over the wire. Client don't need to allocate the entire value when sending and can read the received one progressively. For example a client could send a large file much greater than available ram memory.

At the moment immudb is not yet able to write the data without allocating the entire received object, but in the next release it will be possible a complete communication without allocations. The maximum size of a transaction sent with streams is temporarily limited to a payload of 32M.

Supported stream method now available in the SDK are:


# Chunked reading

It's possible to read returned value chunk by chunk if needed. This grant to the clients capabilities to handle data coming from immudb chunk by chunk

# User management

User management is exposed with following methods:

  • CreateUser
  • ChangePermission
  • ChangePassword

Password must have between 8 and 32 letters, digits and special characters of which at least 1 uppercase letter, 1 digit and 1 special character.

Admin permissions are:

  • PermissionSysAdmin = 255
  • PermissionAdmin = 254

Non-admin permissions are:

  • PermissionNone = 0
  • PermissionR = 1
  • PermissionRW = 2

# Multiple databases

Starting with version 0.7.0 of immudb, we introduced multi-database support. By default, the first database is either called defaultdb or based on the environment variable IMMUDB_DBNAME. Handling users and databases requires the appropriate privileges. Users with PermissionAdmin can control everything. Non-admin users have restricted permissions and can read or write only their databases, assuming sufficient privileges.

Each database has default MaxValueLen and MaxKeyLen values. These are fixed respectively to 1MB and 1KB. These values at the moment are not exposed to client SDK and can be modified using internal store options.

# Index Cleaning

It's important to keep disk usage under control. CleanIndex it's a temporary solution to launch an internal clean routine that could free disk space.

immudb uses a btree to index key-value entries. While the key is the same submitted by the client, the value stored in the btree is an offset to the file where the actual value as stored, its size and hash value. The btree is keep in memory as new data is inserted, getting a key or even the historical values of a key can directly be made by using a mutex lock on the btree but scanning by prefix requires the tree to be stored into disk, this is referred as a snapshot. The persistence is implemented in append-only mode, thus whenever a snapshot is created (btree flushed to disk), updated and new nodes are appended to the file, while new or updated nodes may be linked to unmodified nodes (already written into disk) and those unmodified nodes are not rewritten. The snapshot creation does not necessarily take place upon each scan by prefix, it's possible to reuse an already created one, client can provide his requirements on how new the snapshot should be by providing a transaction ID which at least must be indexed (sinceTx). After some time, several snapshots may be created (specified by flushAfter properties of the btree and the scan requests), the file backing the btree will hold several old snapshots. Thus the clean index process will dump to a different location only the latest snapshot but in this case also writing the unmodified nodes. Once that dump is done, the index folder is replaced by the new one. While the clean process is made, no data is indexed and there will be an extra disk space requirement due to the new dump. Once completed, a considerable disk space will be reduced by removing the previously indexed data (older snapshots). The btree and clean up process is something specific to indexing. And will not lock transaction processing as indexing is asynchronously generated.

# HealthCheck

HealthCheck return an error if immudb status is not ok.

# Immudb SDKs examples

Examples in multiple languages can be found at following links: