Using cloud services is a common practice for backend application architectures. It is important to have control over the data being sent to and from the cloud. Temporal Cloud provides options to ensure secure encryption of data. In this post, we will explore how Temporal Cloud interacts with your infrastructure by default and how you can customize it to prevent any sensitive data from being sent to the cloud.
Understanding Temporal Cloud
To better understand how Temporal Cloud manages data, we have created an image that visualizes the data flow. When Temporal Workers execute a Workflow, the data is sent to the Temporal Cloud and then returned to your server or service upon completion.
Ensuring Security and Encryption
To ensure security and encryption of the data sent to and from the Temporal Cloud, you can define your own DataConverter option for your Temporal Client. Temporal provides a converter library in Go that allows you to create a custom DataConverter. This DataConverter extends a PayloadCodec interface with two methods: Encode and Decode. These methods are responsible for encoding and decoding the data that passes through the Temporal Workers and the Temporal Client.
Creating Your Own DataConverter
When creating your Temporal Client, you have the option to create your own DataConverter. You can define a method called NewEncryptionDataConverter where you define your own Codec to be used in the data conversion process. The Codec struct implements the PayloadCodec interface and uses AES Crypt security.
The Encode method in the Codec struct encodes the data using the specified key and returns a struct following the commonpb.Payload structure. The Decode method decodes the data using the key and returns it in the same protobuf style.
Adding a DataStore to Your DataConverter
In addition to encryption, you can use your DataConverter to store the data in a separate datastore that you host. This can be a MongoDB, SQL DB, Redis, S3, or any other datastore of your choice. The Encode and Decode methods need to be updated to include the datastore dependency.
In the Encode method, you can create a unique ID, insert the record in your datastore using the ID as the primary key, and send the encrypted ID to the Temporal Cloud. In the Decode method, you retrieve the record from the datastore using the decrypted ID and return the data as a commonpb.Payload type.
Conclusion
By customizing your DataConverter, you can add a layer of security to your data and store it in your own datastore instead of sending it to the Temporal Cloud. This allows you to meet regulatory or security requirements that prevent you from sending data outside of your infrastructure. You can find sample code and more information in the GitHub repository provided.
Source link
