Data BlobEntry=>Blob#
Many BlobEntrys from various nodes in the graph can reference the Blob. A blob is simply just binary data that is uniquely identified through a uuid. This approach allows for distributed usage over a large networked system including limited connectivity situations.
Tip
BlobEntrys and Blobs are separate separate by related objects. Various nodes on the graph can have any number of BlobEntrys (except for duplicate blobentry.labels). A user more likely to work the BlobEntrys, and only pull bandwidth heavy binary Blobs to differemt parts of the system network when needed. Similarly, these data blobs can be readily cached across the network, since blobs are immutable.
What is a BlobEntry#
A blob entry is a small amount of structed data that holds reference information to find an actual binary blob which is possibly a massive amount of data. BlobEntrys are plentiful, lightweight, and allow users to more easily organize, find, share, distribute, the contextual information about a data blob over a divergent network how a Blob is associated with the graph. The primary method for storing additional large data as a Blob, however, a blob just “barcoded” binary blob of data with little context or meaning.
Listing BlobEntries on a Node#
The list verb is purposefully limited to just returning a list of labels (i.e. List[string]), and therefore the following call only returns the labels of BlobEntrys attached a particular node (variable, session, robot, etc.):
ent_labels = listBlobEntries(fgclient, 'x0')
# ent_labels = await listBlobEntriesAsync(fgclient, 'x0')
# human friendly labels
print('Human friendly BlobEntry labels on x0 are', ent_labels)
# e.g. ['LEFTCAM_1002', 'LEFTCAM_1003', 'UserCalib', 'GPS', 'JoystickCmds']
Getting a BlobEntry#
Using the human friendly label from the listBlobEntries above, we can fetch the BlobEntry
entry = getBlobEntry(fgclient, 'LEFTCAM_1002')
# entry = await getBlobEntryAsync(fgclient, 'LEFTCAM_1002')
The entry object is well structured
A BlobEntry is a small about of structured data that holds reference information to find an actual blob. Many BlobEntry`s can exist on different graph nodes spanning Robots, and Sessions which can all reference the same `Blob. A BlobEntry is also a equivalent to a bridging entry between local .originId and a remotely assigned .blobIds.
All .blobId`s are unique across the entire distributed system and are immutable. The `.originId should be globally unique except for stochastic uuid4 collisions that cannot be checked from a main reference owing to practical limitations such as network connectivity.
- Parameters:
id (Optional[UUID]) – Remotely assigned and globally unique identifier for the BlobEntry itself (not the .blobId).
label (str) – Human friendly label of the Blob and also used as unique identifier per node on which a BlobEntry is added. E.g. do “LEFTCAM_1”, “LEFTCAM_2”, … of you need to repeat a label on the same variable.
blobId (Optional[UUID]) – Machine friendly and globally unique identifier of the ‘Blob’, usually assigned from a common point in the system. This can be used to guarantee unique retrieval of the large data blob.
originId (UUID) – Machine friendly and locally assigned identifier of the ‘Blob’. .originId`s are mandatory upon first creation at the origin regardless of network access. Separate from `.blobId since some architectures do not allow edge processes to assign a uuid to data store elements.
timestamp (datetime) – When the Blob itself was first created.
description (str) – Additional information that can help a different user of the Blob.
blobstore (str) – A hint about where the Blob itself might be stored. Remember that a Blob may be duplicated over multiple blobstores.
hash (str) – A hash value to ensure data consistency which must correspond to the stored hash upon retrieval. [Legacy: some usage functions allow the check to be skipped if needed.]
mimeType (str) – MIME description describing the format of binary data in the Blob, e.g. ‘image/png’ or ‘application/json; _type=CameraModel’.
origin (str) – Context from which a BlobEntry=>Blob was first created. E.g. user|robot|session|varlabel.
metadata (dict) – Additional storage for functional metadata used in some scenarios, e.g. to support advanced features such as parsejson(base64decode(entry.metadata))[‘time_sync’].
createdTimestamp (Optional[datetime]) – When the BlobEntry was created.
lastUpdatedTimestamp (Optional[datetime]) – Use carefully, but necessary to support advanced usage such as time synchronization over Blob data.
_type (str) – Self type declaration for when duck-typing happens.
_version (str) – Type version of this BlobEntry.
userLabel (Optional[str]) –
robotLabel (Optional[str]) –
sessionLabel (Optional[str]) –
variableLabel (Optional[str]) –
factorLabel (Optional[str]) –
Getting the associated Blob#
A binary Blob is basically just a “barcoded” piece of data that can be associated (via individual BlobEntrys) multiple times across multiple graph nodes, sessions, or robots.
Data blobs can be fetched using the unique .blobId as primary (or .originId as secondary) reference. Also note that the blob itself may also be replicated across any number of blob stores, depending on the application:
blob = getBlob(fgclient, entry.blobId]; checkhash=false) # legacy versions did not use .hash check
# blob = await getBlobAsync(fgclient, entry.blobId]; checkhash=false)
The blob contains binary information, for example this mimeType = application/octet-stream/json; _type=ROS.sensor_msgs.NavSatFix:
b'{"latitude":41.7325,"altitude":2.211,"header":{"stamp":{"secs":1670378554,"nsecs":000624417},"seq":91,"frame_id":"gps","_type":"ROS1/std_msgs/Header"},"status":{"status":0,"service":1},"position_covariance":[0.265225,0.0,0.0,0.0,0.265225,0.0,0.0,0.0,0.556516],"longitude":-49.946944,"_type":"ROS1/sensor_msgs/NavSatFix","position_covariance_type":2}'
Tip
A blob is owned by a user and only accessible by other users if allowed via approved roles or permissions.
Tip
BlobEntry.hash helps ensure data consistency by rehasing the retrieved blob binary data itself.
Adding BlobEntries#
Warning
Adding Blob or BlobEntrys from the Python SDK are under construction and expected to be part of the v0.6.1 release. This functionality has already been released with the JuliaLang SDK.
Blobs can be linked to any variable (future node) in the graph. This is easily done by adding a BlobEntry:
res = addBlobEntry(fgclient, 'x12', entries[1].id, entries[1].label, len(blob), entries[1].mimeType)
# res = await addBlobEntryAsync(fgclient, 'x12', entries[1].id, entries[1].label, len(blob), entries[1].mimeType)
Adding New Blobs#
Warning
Adding Blob or BlobEntrys from the Python SDK are under construction and expected to be part of the v0.6.1 release. This functionality has already been released with the JuliaLang SDK.
It is also possible to push data blobs:
blobId = addBlob(fgclient, "testimage.png", imgblob)
Remember to add at least one BlobEntry somewhere in your session so that you might find it again in the future, see addBlobEntry above.