# SDKs api
# Contents
- Connection and authentication
- State management
- Tamperproof reading and writing
- Writing and reading
- History
- Counting
- Scan
- References
- Secondary indexes
- Transactions
- Tamperproofing utilities
- User management (ChangePermission,SetActiveUser,DatabaseList)
- Multiple databases(CreateDatabase,UseDatabase)
- Index cleaning
- Health
- Immudb SDKs examples
# Connection and authentication
The immudb server runs on port 3323 as the default. The code examples below illustrate how to connect your client to the server and authenticate using default options and the default username and password.
You can modify defaults on the immudb server in immudb.toml
in the config folder.
# Mutual TLS
To enable mutual authentication, a certificate chain must be provided to both the server and client. That will cause each to authenticate with the other simultaneously. In order to generate certs, use the openssl tool: generate.sh (opens new window).
./generate.sh localhost mysecretpassword
This generates a list of folders containing certificates and private keys to set up a mTLS connection.
# Disable authentication
You also have the option to run immudb with authentication disabled. However, without authentication enabled, it's not possible to connect to a server already configured with databases and user permissions. If a valid token is present, authentication is enabled by default.
# State management
It's the responsibility of the immudb client to track the server state. That way it can check each verified read or write operation against a trusted state.
# Verify state signature
If immudb
is launched with a private signing key, each signed request can be verified with the public key.
This ensures the server identity.
Check state signature to see how to generate a valid key.
# Tamperproof reading and writing
You can read and write records securely using built-in cryptographic verification.
# Verified get and set
The client implements the mathematical validations, while your application uses a traditional read or write function.
# Writing and reading
The format for writing data is the same in both Set and VerifiedSet, as is the same for reading data in both Get and VerifiedGet.
The only difference is that VerifiedSet returns proofs needed to mathematically verify that the data was not tampered. Note that generating that proof has a slight performance impact, so primitives are allowed without the proof. It is still possible get the proofs for a specific item at any time, so the decision about when or how frequently to do checks (with the Verify version of a method) is completely up to the user. It's possible also to use dedicated auditors to ensure the database consistency, but the pattern in which every client is also an auditor is the more interesting one.
# Get and Set
# Get at and since a transaction
You can retrieve a key on a specific transaction with VerifiedGetAt
and since a specific transaction with VerifiedGetSince
.
# Transaction by index
It's possible to retrieve all the keys inside a specific transaction.
# Verified transaction by index
It's possible to retrieve all the keys inside a specific verified transaction.
# History
The fundamental property of immudb is that it's an append-only database. This means that an update does not change an existing record. Instead, it is a new insert of the same key with a new value. It's possible to retrieve all the values for a particular key with the history command.
History
accepts the following parameters:
Key
: a key of an itemOffset
: the starting index (excluded from the search). OptionalLimit
: maximum returned items. OptionalDesc
: items are returned in reverse order. OptionalSinceTx
:
# Counting
Counting entries is not supported at the moment.
# Scan
The scan
command is used to iterate over the collection of elements present in the currently selected database.
Scan
accepts the following parameters:
Prefix
: prefix. If not provided all keys will be involved. OptionalSeekKey
: initial key for the first entry in the iteration. OptionalDesc
: DESC or ASC sorting order. OptionalLimit
: maximum returned items. OptionalSinceTx
: immudb will wait that the transaction provided by SinceTx be processed. OptionalNoWait
: when true scan doesn't wait that txSinceTx is processed. Optional
Having the possibility to get data specifying a transaction id:
AtTx
, it’s the optimal way to retrieve the data, as it can be done with random access to it. And it can be made immediately after the transaction was committed or at any point in the future. When the transaction ID is unknown by the application and the query is made by key or key prefixes, it will be served through the index, depending on the insertion rate, it can be delayed or up to date with inserted data, using a big number inSinceTx
withNoWait
in true will mean that the query will be resolved by looking at the most recent indexed data, but if your query needs to be resolved after some transactions has been inserted, you can setSinceTx
to specify up to which transaction the index has to be made for resolving it.
# References
SetReference
is like a "tag" operation. It appends a reference on a key/value element.
As a consequence, when we retrieve that reference with a Get
or VerifiedGet
the value retrieved will be the original value associated with the original key.
Its VerifiedReference
counterpart is the same except that it also produces the inclusion and consistency proofs.
# SetReference and VerifiedSetReference
# GetReference and VerifiedGetReference
When reference is resolved with get or verifiedGet in case of multiples equals references the last reference is returned.
# Resolving reference with transaction id
It's possible to bind a reference to a key on a specific transaction using SetReference
and VerifiedSetReferenceAt
# Secondary indexes
On top of the key value store immudb provides secondary indexes to help developers to handle complex queries.
# Sorted sets
The sorted set
data type provides simplest secondary index you can create with immudb. That's a data structure that represents a set of elements ordered by the score of each element, which is a floating point number.
The score type is a float64 to accommodate the maximum number of uses cases.
64-bit floating point gives a lot of flexibility and dynamic range, at the expense of having only 53-bits of integer.
When an integer64 is cast to a float there could be a loss of precision, but the insertion order is guaranteed by the internal database index that is appended to the internal index key.
ZAdd
can reference an item by key
or by index
.
ZScan
accepts following arguments:
Set
: the name of the collectionSeekKey
: initial key for the first entry in the iteration. OptionalSeekScore
: the min or max score for the first entry in the iteration, depending on Desc value. OptionalSeekAtTx
: the tx id for the first entry in the iteration. OptionalInclusiveSeek
: the element resulting from the combination of theSeekKey
SeekScore
andSeekAtTx
is returned with the result. OptionalDesc
: DESC or ASC sorting order. OptionalSinceTx
: immudb will wait that the transaction provided by SinceTx be processed. OptionalNoWait
: when true scan doesn't wait that txSinceTx is processed. OptionalMinScore
: minimum score filter. OptionalMaxScore
: maximum score filter. OptionalLimit
: maximum number of returned items. Optional
Having the possibility to get data specifying a transaction id:
AtTx
, it’s the optimal way to retrieve the data, as it can be done with random access to it. And it can be made immediately after the transaction was committed or at any point in the future. When the transaction ID is unknown by the application and the query is made by key or key prefixes, it will be served through the index, depending on the insertion rate, it can be delayed or up to date with inserted data, using a big number inSinceTx
withNoWait
in true will mean that the query will be resolved by looking at the most recent indexed data, but if your query needs to be resolved after some transactions has been inserted, you can setSinceTx
to specify up to which transaction the index has to be made for resolving it.
# Transactions
GetAll
, SetAll
and ExecAll
are the foundation of transactions in immudb. They allow the execution of a group of commands in a single step, with two important guarantees:
- All the commands in a transaction are serialized and executed sequentially. No request issued by another client can ever interrupt the execution of a transaction. This guarantees that the commands are executed as a single isolated operation.
- Either all of the commands are processed, or none are, so the transaction is also atomic.
# GetAll
# SetAll
A more versatile atomic multi set operation
SetBatch and GetBatch example
# ExecAll
ExecAll
allows multiple insertions at once. The difference is that it is possible to specify a list of mixes of key/value sets, references and zAdd insertions.
The argument of a ExecAll is an array of the following types:
It's possible to persist and reference items that are already persisted on disk. In that case is mandatory to provide the index of the referenced item. This has to be done for:
Op_ZAdd
Op_Ref
IfzAdd
orreference
is not yet persisted on disk it's possible to add it as a regular key value and the reference is done onFly. In that case ifBoundRef
is true the reference is bounded to the current transaction values.
# Tx Scan
TxScan
permits iterating over transactions.
The argument of a TxScan
is an array of the following types:
InitialTx
: initial transaction idLimit
: number of transactions returnedDesc
: order of returned transacations
# Tamperproofing utilities
# Current State
CurrentState
returns the last state of the server.
# User management
User management is exposed with following methods:
- CreateUser
- ChangePermission
- ChangePassword
Password must have between 8 and 32 letters, digits and special characters of which at least 1 uppercase letter, 1 digit and 1 special character.
Admin permissions are:
- PermissionSysAdmin = 255
- PermissionAdmin = 254
Non-admin permissions are:
- PermissionNone = 0
- PermissionR = 1
- PermissionRW = 2
# Multiple databases
Starting with version 0.7.0 of immudb, we introduced multi-database support.
By default, the first database is either called defaultdb
or based on the environment variable IMMUDB_DBNAME
.
Handling users and databases requires the appropriate privileges.
Users with PermissionAdmin
can control everything. Non-admin users have restricted permissions and can read or write only their databases, assuming sufficient privileges.
Each database has default MaxValueLen and MaxKeyLen values. These are fixed respectively to 1MB and 1KB. These values at the moment are not exposed to client SDK and can be modified using internal store options.
# Index Cleaning
It's important to keep disk usage under control. CleanIndex
it's a temporary solution to launch an internal clean routine that could free disk space.
immudb uses a btree to index key-value entries. While the key is the same submitted by the client, the value stored in the btree is an offset to the file where the actual value as stored, its size and hash value. The btree is keep in memory as new data is inserted, getting a key or even the historical values of a key can directly be made by using a mutex lock on the btree but scanning by prefix requires the tree to be stored into disk, this is referred as a snapshot. The persistence is implemented in append-only mode, thus whenever a snapshot is created (btree flushed to disk), updated and new nodes are appended to the file, while new or updated nodes may be linked to unmodified nodes (already written into disk) and those unmodified nodes are not rewritten. The snapshot creation does not necessarily take place upon each scan by prefix, it's possible to reuse an already created one, client can provide his requirements on how new the snapshot should be by providing a transaction ID which at least must be indexed (sinceTx). After some time, several snapshots may be created (specified by flushAfter properties of the btree and the scan requests), the file backing the btree will hold several old snapshots. Thus the clean index process will dump to a different location only the latest snapshot but in this case also writing the unmodified nodes. Once that dump is done, the index folder is replaced by the new one. While the clean process is made, no data is indexed and there will be an extra disk space requirement due to the new dump. Once completed, a considerable disk space will be reduced by removing the previously indexed data (older snapshots). The btree and clean up process is something specific to indexing. And will not lock transaction processing as indexing is asynchronously generated.
# HealthCheck
HealthCheck return an error if immudb
status is not ok.
# Immudb SDKs examples
Examples in multiple languages can be found at following links: