Ubiq Library for MaxCompute

The Ubiq Security MaxCompute library provides convenient interaction with the Ubiq Security Platform API from applications written in the Java language. It includes a pre-defined set of classes that will provide simple interfaces to encrypt and decrypt data.

Documentation

See the MaxCompute API docs.

Before You Begin

In order to use the Ubiq Security MaxCompute library, you will need to configure your MaxCompute Workspace for internet connectivity. The library will make web calls to retrieve keys and dataset definitions. By default, the internet is not accessible in a MaxCompute enviornment, but it is possible with configuration. Below are the steps to whitelist the Ubiq domain in your Workspace. Alternatively, Alibaba's documentation can be found here: MaxCompute | Network Connection Process - Access over the Internet

1. From the MaxCompute Console, in the left-side navigation pane, click Workspace, then Projects.
2. On the project you want to use Ubiq with, select Manage from the Actions on the right.
3. On the bottom of the Parameter Configuration pane, there should be a section titled MaxCompute External Network. (If this is not there, contact Support.) Click Edit, then add api.ubiqsecurity.com and port 443 in the boxes.
4. Hit Submit.

MaxCompute will perform verification on the domain.

Note: We have submitted it before and been approved so it may work first try in your environment. If not, you will need to Submit an Application to add it. (MaxCompute may display an alternative support form link after failure. You can try that as well, but Alibaba do not seem to notify you on success/failure via that route.)

Installation

The library needs to be configured with your account credentials which is available in your [Ubiq Dashboard][dashboard] [credentials][credentials].

Build the Ubiq Jar

A fork of the Ubiq Java library has been made for compatibility with MaxCompute. This is in the /ubiq-maxcompute-java directory of the Ubiq MaxCompute public repository. A jar will need to be built for use with the UDFs.

Use the following command to use [gradlew] to build the JAR file

# Linux / Mac
./gradlew assemble build

# windows
.\gradlew assemble build

Build the UDF Jar

You will need to make a Jar file containing the methods you wish to use. We have provided a base in the /ubiq_udf folder.

The UbiqBase setup method handles adding the certificate needed for a secure SSL connection to the Ubiq API, and then creates the Ubiq Credentials and EncryptDecrypt objects. UbiqEncrypt and UbiqDecrypt extend this with the appropriate method calls in evaluate.

When calling a UDF, MaxCompute will call setup once per query, and then evaluate for each row the method is called on. It is recommended for anything you wish to reuse to be initialized as a class variable, ideally done only once in setup.

Dependencies

The following is a list of the JAR files required to compile, test, or deploy the library. Note that the MaxCompute JVM does not contain a modern Root CA pack. The below SSL Certificate pack that includes the Let's Encrypt Root CA's is included so provide support for the SSL certs used by ubiqsecurity.com from the MaxCompute environment.

The MaxCompute UDF Library:

• odps-sdk-udf-0.43.3-public.jar

UbiqSecurity Dependencies:

• bcprov-jdk18on-1.76.jar
• bcutil-jdk18on-1.76.jar
• bcpkix-jdk18on-1.76.jar
• gson-2.10.jar
• guava-18.0.jar
• commons-codec-1.11.jar
• commons-logging-1.2.jar
• jcommander-1.78.jar
• json-simple-1.1.1.jar
• junit-4.13.1.jar

SSL Certificate

• The ISRG Root X1 DER file Let's Encrypt - Certificates isrgrootx1.der

Package the Jar

Complile the classes with javac, then make the jar with jar.

javac -cp "/ubiq-maxcompute-java/build/libs/*:/path/to/dependencies/*" ubiq_udf/*.java
jar -cvf ubiq_udf.jar ubiq_udf/*.class

Upload to MaxCompute

If you're using MaxCompute Studio, in the Project Explorer, click the Puzzle icon (Add Resource), then add all the dependencies, the ubiqsecurity jar, the ubiq_udf.jar , and the SSL Certificate (isrgrootx1.der).

If you're using MaxCompute's ODPS, use ADD JAR <filepath> and ADD RESOURCE <filepath> to add all of the files.

Initialize the UDF

Run the following queries to create the UDFs in your project.

CREATE FUNCTION ubiq_encrypt AS 'ubiq_udf.UbiqEncrypt' USING 'bcpkix-jdk18on-1.76.jar,guava-18.0.jar,junit-4.13.1.jar,commons-logging-1.2.jar,bcprov-jdk18on-1.76.jar,bcutil-jdk18on-1.76.jar,commons-codec-1.11.jar,json-simple-1.1.1.jar,gson-2.10.jar,jcommander-1.78.jar,isrgrootx1.der,letsencrypt.bks,ubiq-gestalt.jar,ubiqsecurity-2.2.6.jar';

CREATE FUNCTION ubiq_decrypt AS 'ubiq_udf.UbiqDecrypt' USING 'bcpkix-jdk18on-1.76.jar,guava-18.0.jar,junit-4.13.1.jar,commons-logging-1.2.jar,bcprov-jdk18on-1.76.jar,bcutil-jdk18on-1.76.jar,commons-codec-1.11.jar,json-simple-1.1.1.jar,gson-2.10.jar,jcommander-1.78.jar,isrgrootx1.der,letsencrypt.bks,ubiq-gestalt.jar,ubiqsecurity-2.2.6.jar';

Usage

Structured Encryption

The below command performs structured encryption by calling the Ubiq API to get Dataset metadata corresponding to the given Dataset name (e.g., 'SSN') and an encryption key.

INSERT INTO my_table (name, secure_data) values ("Jeffrey", ubiq_encrypt("my_dataset", "secure_data", <accessKeyId>, <secretSigningKey>, <secretCryptoAccessKey>);

Structured Decryption

The below command performs structured decryption by calling the Ubiq API to get Dataset metadata corresponding to the given Dataset name (e.g., 'SSN') and a corresponding key.

SELECT name, ubiq_decrypt("my_dataset", secure_data, <accessKeyId>, <secretSigningKey>, <secretCryptoAccessKey>) FROM my_table;

Ubiq API Error Reference

Occasionally, you may encounter issues when interacting with the Ubiq API.