Introduction

A Dataset is the primary building block (in the Ubiq Dashboard) of data that you choose to encrypt.

Datasets can be configured as two types:

1. Structured

Example: Data stored in a database column with a fixed length and type. Like a name, address, or SSN.

2. Unstructured

Example: Files (audio, video, PDF, text, etc.) stored in an unstructured data store such as AWS S3, Google Cloud Storage, or a Data Lake.

Given an application could have multiple data elements and data types that you’d like to encrypt, Datasets provide you a more logical and flexible representation of each.

Create a Dataset

Prepare a secure location for storage of Ubiq API Key Credentials. The process of creating a Dataset will create cryptographic API Key Credentials for your application that will only be shown once in the Ubiq UI. To ensure confidentiality of encrypted data, it is important to keep these API Key Credentials secret. They should not be stored in standard files or checked into source code repositories. Additionally, the availability of these API Key Credentials is paramount. If lost or destroyed, they cannot be restored and data encrypted with those Credentials may be irrecoverable.

❗️
To ensure security of API Key Credentials, they should be stored in a well-managed and backed up secret management server or password vault.

On the left side menu click Datasets.

The Datasets panel appears.

Click on the + New Dataset button to enter the Dataset Creation Wizard.

Input the following Information:
a) Dataset Name - An internal name that will be used to help you identify what data you're encrypting
b) Description - A short description to keep track of your dataset definitions
c) Tags - Tags can be used to mark a dataset's purpose or intended audience.
d) Primary Key - Create a new Primary Key, or manually select an existing one
e) Click Continue to input the Data Type that you will be encrypting

Enter Data Type information.
1. Select Structured or Unstructured
  
  Structured has multiple sub-types available for commonly used data patterns. Choose the one that best suits your data. (Most common is Formatted String.) See Structured Data Types below for what each one is for.
  
  Depending on your selected Data Type, there may be varying configuration afterwards.
2. Click Continue to go to the next step

If you selected Unstructured Data, then you will skip Step 8 and go straight to Step 9 to review. If you selected Structured Data and the Formatted String type, you will be presented with the Formated String Definition page.

📘
For the first time creating a Structured Formatted String Dataset, here are some suggested values for the variables:
Example: A U.S. Social Security Number (SSN)
Input character set: 0123456789
Output character set: 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
Passthrough: - (dash & space)
Min Input Length: 9
Max Input Length: 9

Review all the setting for your New Dataset.

Click Create and your new Dataset will be displayed on the Dataset Panel.

Structured Data Types

When you create a Structured dataset, choose the data type that matches the data you are protecting. The type controls which options the Dataset Creation Wizard asks for, and how the Ubiq libraries encrypt and decrypt the value.

For Integer, Date and Date & Time datasets, the encrypted value is the same type as the original: an integer encrypts to an integer, and a date to a date. Protected data can stay in its existing column type, with no custom conversion code in your application.

Data Type	Use it for	Options in the wizard	Supported input
Formatted String	Well-defined strings, such as an SSN or credit card number	Input and output character sets, min and max input length, and optional passthrough characters, prefix and suffix for partial encryption	Set by your min and max input length
Generic String	Strings of unknown length or format	Input and output character sets, min and max input length, and optional Input Encoding (Base32 or Base64) and Input Padding Character	Set by your min and max input length
Token	A fixed-length, tokenized value (base62: A-Z, a-z, 0-9)	Token length: 64 or 128 characters	64-character token: strings under 40 characters 128-character token: strings under 80 characters
Integer	Whole numbers	Data size: 32-bit or 64-bit	32-bit: -99,999,999 to 99,999,999 64-bit: -9,999,999,999,999,999 to 9,999,999,999,999,999
Date	Calendar dates with no time component	None	01/01/0001 to 11/28/2738
Date & Time	Timestamps, to the second	None	1653-02-10 06:13:21 to 2286-11-20 17:46:39 UTC

Encrypted integers stay within -1,470,375,551 to 1,470,375,551 (32-bit) and -2,032,385,242,251,560,000 to 2,032,385,242,251,560,000 (64-bit). Encrypted Date & Time values fall between 0018-01-16 11:01:21 and 3921-12-16 12:58:39 UTC. Check these ranges against your column types before you encrypt data in place.

📘
Generic String padding and encoding
Use the Input Padding Character when values can be shorter than the minimum length encryption needs. Short values are padded before they are encrypted, and the padding is removed on decryption. Use Input Encoding (Base32 or Base64) for data whose characters don't fit a practical input character set. The libraries encode the value before encryption and decode it after decryption, so your application always works with the original value.

Integer, Date and Date & Time datasets use each library's typed methods, such as encryptInt / encryptLong / encryptDate in Java or CipherInt64 / CipherDate / CipherDateTime in Go. Formatted String, Generic String and Token datasets use the standard structured encrypt and decrypt methods. The data types are supported from C/C++ 2.3.0.0, .NET 2.4.0, Go 2.3.0, Java 2.3.0, Node.js 2.4.0, PHP 2.1.0 and Python 2.4.0. See each library guide for examples.