Datasets

Logical groupings of resources and time-series. Datasets can be nested (a dataset can belong to a parent dataset), and that hierarchy is live in queries: filtering time-series by a dataset also matches everything beneath it. Filter series →

The hierarchy is built from BELONGS_TO edges, and the server enforces that: a relation pointing at a dataset must be BELONGS_TO, and a dataset can only claim a time-series that isn't already in another dataset. Edge rules →

External ids are stored exactly as you send them

The server does not rewrite a dataset external id: Plant-A stays Plant-A. Some clients derive one from the name as a convenience (the Rust Dataset::new below), and that derivation is snake_case — but it is a client-side default, not a server rule.

Uniqueness and lookup both ignore case, so plant_a collides with PLANT_A and either spelling finds the same dataset. Datasets are also subject to the naming policy if an administrator has set one. External ids & naming →

Create

Java

DataSetModel dataset = new DataSetModel();
dataset.setExternalId("plant_a");
dataset.setName("Plant A");

client.datasets().create(List.of(dataset));

Python

import intellistream_datahub_sdk

dataset = intellistream_datahub_sdk.Dataset(external_id="plant_a", name="Plant A")
client.datasets.create([dataset])

Rust

use intellistream_datahub_sdk::datasets::Dataset;

// external_id is derived as snake_case of the name → "plant_a"
let dataset = Dataset::new("Plant A".into());
api.datasets.create(&vec![dataset]).await?;

Look up & delete

Java

DataWrapper<DataSetModel> some = client.datasets()
        .byIds(List.of(IdCollection.createFromExternalId("plant_a")));

client.datasets().delete(List.of(IdCollection.createFromExternalId("plant_a")));

Python

some = client.datasets.by_ids(["plant_a"])
client.datasets.delete(["plant_a"])

Rust

use intellistream_datahub_sdk::generic::IdAndExtId;

let some = api.datasets.by_ids(&vec![IdAndExtId::from_external_id("plant_a")]).await?;
api.datasets.delete(&vec![IdAndExtId::from_external_id("plant_a")]).await?;

Filter

POST /datasets/filter finds data sets by structured criteria, combined with AND. It is exactly the criteria every node type shares — a data set has no dataSetId of its own, being the thing other nodes are scoped by:

Criterion	Matching
id, externalId, name, source	Patterns, case-insensitive. * and % are wildcards, _ is literal, and an entry with no wildcard matches exactly.
labels	Data sets carrying all of these labels.
metadata	Every key/value pair given must be present. A null value matches the key alone, whatever it holds.
createdTime, lastUpdatedTime	{ "min": …, "max": … } bounds.

Each field except labels and metadata takes either a bare value or an array, whose entries are combined with OR — which is why they are named in the singular. limit defaults to 1000 and is capped at 10000, and the page can be ordered and walked exactly as timeseries can.

POST /datasets/list is the same handler with an empty filter, so it returns everything your token may read.

Java

DataSetFilter criteria = new DataSetFilter();
criteria.setName(List.of("Plant *"));
criteria.setMetadata(Map.of("tier", "gold"));

DataWrapper<DataSetModel> matches = client.datasets().filter(criteria);

Pass a DataSetRetreiver instead of the bare criteria to set limit, sort or cursor.

Python

matches = client.datasets.filter(intellistream_datahub_sdk.DatasetFilter(
    intellistream_datahub_sdk.BasicDatasetFilter(name="Plant *", metadata={"tier": "gold"}),
    limit=100))

Rust

use intellistream_datahub_sdk::datasets::{BasicDatasetFilter, DatasetFilter};

let criteria = BasicDatasetFilter::new()
    .set_name(vec!["Plant *".to_string()])
    .build();
let matches = api.datasets.filter(&DatasetFilter::from_filter(criteria)).await?;

There is no writeProtected or deactivated

Both were removed server-side as inert. The api drops unknown keys silently, so a filter still carrying one looked like it was narrowing and was not.

Access control

Access to a data set is administered in Keycloak (or the directory behind it), not in DataHub. A grant is membership of an organization group, scoped to one organization:

Group	Grants
/datasets/<externalId>/read	Read everything in that data set, and in every data set beneath it.
/datasets/<externalId>/write	Write, with the same inheritance.
/datasets/*/read	Read every data set.
/datasets/*/write	Write every data set.

Read and write are independent: a write grant does not imply read, the wildcard included. The DATAHUB_ADMIN realm role grants read and write to everything (an operator escape hatch). Entities outside any data set follow the wildcard too: reading them needs /datasets/*/read, writing or creating them needs /datasets/*/write (or admin).

Two consequences worth knowing when you code against this:

• A missing grant is a 403 with an application/problem+json body naming the dataSetId and the permission (read or write) you lack. List, filter and search endpoints never 403 on grants: rows in data sets you cannot read are silently omitted instead.
• Managing a data set itself is stricter. Creating, updating or deleting a data set (as opposed to the data in it) requires the /datasets/*/write grant or DATAHUB_ADMIN; grants on individual data sets are never enough, deliberately: a data set is the unit access is granted on, so renaming or re-parenting one changes what existing grants cover. The 403 detail spells this out.

The API reads grants from the identity provider's UserInfo endpoint, not from the token, so a changed grant takes effect within about a minute, without a new token.

What each client covers

Operation	Java	Python	Rust
Create	datasets().create	datasets.create	datasets.create
Look up by id / external id	datasets().byIds	datasets.by_ids	datasets.by_ids
List	datasets().list	datasets.list	datasets.list
Filter	datasets().filter	datasets.filter	datasets.filter
Search	datasets().search	datasets.search	datasets.search
Update	datasets().update	datasets.update	datasets.update
Delete	datasets().delete	datasets.delete	datasets.delete
Access policies	HTTP	datasets.policies	datasets.policies

All three clients now cover the whole surface bar the access-policy read in Java, which still goes through the endpoint directly.